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Abstract 

Consider a "wireless net"work of n nodes represented by a graph G = {V, E) "where an 
edge G E models the fact that transmissions of i and j interfere "with each other, i.e. 
simultaneous transmissions of i and j become unsuccessful. Hence it is required that at 
each time instance a set of non-intcrfcring nodes (corresponding to an independent set in G) 
access the "wireless medium. To utilize wireless resources efficiently, it is required to arbitrate 
the access of medium among interfering nodes properly. Moreover, to be of practical use, 
such a mechanism is required to be totally distributed as "well as simple. 

As the main result of this paper, "we provide such a medium access algorithm. It is 
randomized, totally distributed and simple: each node attempts to access medium at each 
time "with probability that is a function of its local information. Wc establish efficiency of the 
algorithm by sho"wing that the corresponding net"work Markov chain is positive recurrent as 
long as the demand imposed on the net"work can be supported by the "wireless net'work (using 
any algorithm). In that sense, the proposed algorithm is optimal in terms of utilizing "wireless 
resources. The algorithm is oblivious to the net'work graph structure, in contrast "with the so- 
called polynomial hack-off algorithm by Hastad-Leighton-Rogoff (STOC '87, SICOMP '96) 
that is established to be optimal for the complete graph and bipartite graphs (by Goldbcrg- 
MacKenzie (SODA '96, JCSS '99)). 
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1 Introduction 



We consider a single-hop wireless network of n nodes or queues represented hy V = {1, . . . ,n}. 
Time is discrete indexed by r G {0,1,...}. Unit-size packets arrive at queue i as per an 
exogenous process. Let ^^(t) denote the number of packets arriving at queue i in the time slot 
[r, r + 1). For simplicity, we shall assume Aj(-) as an independent Bernoulli process with rate 
Xi, i.e. Xi = F{Ai{T) = 1) and Ai(r) e {0, 1} for all i,r > 00 Let Qi{T) e N be the number of 
packets in queue i at the beginning of time slot [r, r + 1). 

The work from queues is served at the unit rate subject to interference constraints. Specif- 
ically, let G = {V,E) denote the inference graph : {i,j) G E implies that queues i and j 
can not transmit simultaneously since their transmissions interfere with each other. Formally, 
let 0"j(T) G {0, 1} denote whether the queue i is (successfully) transmitting at time r, and let 
o-(r) = [(Jj(r)]. Then, 

cr(T) G X(G) = {p= [p^] G {0, 1}" :p^+pj<l for all (i, j) G E}, for r > 0, 

i.e. I{G) is the set of independent sets of G. The resulting queueing dynamics can be summarized 
as 

Qi{T + 1) = Qiir) - o-i(r)I{Q^(^)>o} + ^^(t), for t > 0, l<i<n. 

Here ^{x} = 1 if 2; ='true' and if x ='false'. 

Now an algorithm, which we shall call medium access, is required to choose ct{t) G X(G) 
at the beginning of each time r. A good medium access algorithm should choose ct{t) so as to 
utilize the wireless medium as efficiently as possible. Putting it another way, it should be able 
to keep queues finite for as large a set of arrival rates A = [Xi] as possible. Towards this, define 
the capacity region 

A = |y G : y < ^ ao-cr with a^- > 0, ^ Oo- < l|. 

creX(G) (TGI{G) 

Since ct{t) G I{G), the effective 'service' rate induced by any algorithm over time is essentially 
in the closure of A. Therefore, a medium access algorithm can be considered optimal, if it can 
keep queues finite, for any A G A. Formally, if the state of the queueing system including the 
algorithm's decisions and queue-sizes can be described as a Markov chain, then the existence 
of a stationary distribution for this Markov chain and its ergodicity effectively implies that the 
queues remain finite. A sufficient condition for this is aperiodicity and positive recurrence of the 
corresponding Markov chain. This motivates the following definition. 

Definition 1 (Optimal) A medium access algorithm is called optimal if for any A G A the 

(appropriately defined) underlying network Markov chain is positive recurrent and aperiodic. 

To be of practical use, medium access algorithm ought to be simple and totally distributed, i.e. 
should use only local information like queue-size, and past collision history. In such an algorithm, 
each node makes the decision to transmit or not on its own, at the beginning of each time slot. 
At the end of the time slot, it learns whether attempted transmission was successful or not (due 
to a simultaneous attempt of transmission by a neighbor). If a node does not transmit, then 
it learns whether any of its neighbors attempted transmission. And, ideally such an algorithm 
should be optimal. 

^The result in this paper extends easily even for (non-Bernoulli) adversarial arrival processes satisfying 
J2l'^s ^'('^) < K{t- s) + w, with (fixed) w < 00, for all < s < t. 
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1.1 Prior Work 



Design of an efficient and distributed medium access algorithm has been of interest since the 
ALOHA algorithm for the radio network [T] and Local Area Networks [15] in the 1970s. Subse- 
quently a variety of the so-called back-off algorithms or protocols have been extensively studied. 
Various negative and positive properties of back-off protocols were established in various works 

[ISllITllIllMlllllIl]. 

Specifically, Hastad, Leighton and Rogoff [9] studied a medium access algorithm in which each 
node or queue attempts transmission at each time with probability that is inversely proportional 
to a polynomial function of the number of successive failures in the most recent past. They 
established it to be optimal when the interference graph G is complete, or equivalently all 
nodes are competing for one resource (as in the classical Ethernet/LAN). The optimality of this 
polynomial back-off algorithm was further established for G when it is induced by matching 
constraints in a bipartite graph by Goldberg and MacKenzie [7]- However, the optimality of 
polynomial back-off or any other totally distributed medium access algorithm remained open 
for general graphs. The interested reader may find a good summary of results until 2001, on 
medium access on a webpage maintained by Goldberg [8] . 

In the past year or so, significant progress has been made towards this open question. Specif- 
ically, Rajagopalan, Shah and Shin (RSS) [Ml [19] and Jiang and Walrand (JW) [H] proposed 
two different medium access algorithms that operate in continuous time. The RSS algorithm is 
optimal but requires a bit of information exchange between each pair of neighboring nodes per 
unit time. The JW algorithm was established to have a weaker form of optimality, called 'rate 
stability', by Jiang, Shah, Shin and Walrand [10]. In summary, both algorithms stop short of 
being totally distributed and optimal. Further, both of them operate in continuous time and 
thus effectively avoiding the issue of loss in performance due to contention present in discrete 
time considered in this paper. 

1.2 Our Contribution 

The main result of this paper is a totally distributed medium access algorithm that is optimal 
for any interference graph G. It resolves an important question in distributed computation that 
has been of great practical interest. The proposed medium access algorithm builds on the RSS 
algorithm and overcomes its two key limitations by adapting it to the discrete time and removing 
the need for any information exchange between neighboring nodes. In what follows, we explain 
in detail how we overcome such limitations. 

In the proposed medium access algorithm, each node attempts transmission in each time 
slot based on: (a) whether it managed to successfully transmit in the previous time slot, or 
whether any of its neighbors attempted to transmit in the previous time slot; (b) local queue- 
size and estimation of the "weight" of the neighbors. Given this information, each node in the 
beginning of each time slot attempts transmission with probability depending upon (a) and (b). 
Specifically, if the node was successful in the previous time, it does not transmit in this time 
with probability that is inversely proportional to its own weight that depends on (b). Else if no 
other neighbor attempted transmission in the previous time then a node attempts transmission 
with probability ^. Otherwise, with probability 1, a node does not transmit. 

In such an algorithm, the only seeming non-local information is the estimation of the neigh- 
bors' weight in (b). An important contribution of this work is the design of a non-trivial learning 
mechanism, based only on information of type (a), that estimates the neighbors' weight without 
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any explicit information exchange. We note that, in contrast, the RSS algorithm had required 
explicit information exchange for estimating the neighbors' weight. 

To establish optimality of the proposed algorithm, we show that, in essence, the value of 

(Tj(r) loglogQj(r) is close to ma,Xp^x{G)Y.i Pi^°S^°sQi{'T), on average for all large enough 
T. That is, effectively the distributed medium access chooses o"(r) that is (close to) maximum 
weight independent set of G when node weights are equal to log log of the queue-sizes. Such a 
property is known (cf. (201 [22]) to imply that F{Qi{T)) (where F{x) = JJ^ log logy dy) is a 
potential (or Lyapunov, energy) function for the system state so that the function is expected 
to decrease by at least a fixed amount as long as A S A. This subsequently establishes that the 
network as a Markov chain is positive recurrent (implying the optimality of the algorithm). 

We establish the near optimality of o"i(r) log log Qi^r) under the medium access algorithm 
in two steps. To begin with, we observe that the evolution of cr(r) under the algorithm is a 
Markov chain on the space of independent sets I{G) with time-varying transition probabilities. 
For this Markov chain, at any particular time r, let 7r(r) be the stationary distribution at time 
T (given transition probabilities at time r). 

In the first step, we study this (time- varying, stationary) distribution 7r(r) and show that it 
is approximately 'product-form'. To obtain such an approximate characterization, we show that 
the transition probabilities of the Markov chain are well approximated by those of a reversible 
Markov chain which has a product-form stationary distribution. A novel comparison relation 
between stationary distributions of two Markov chains in terms of the relation between their 
transition probabilities leads to the approximate product-form characterization of 7r(r). We note 
that the RSS algorithm (and similarly, the JW algorithm) had used the continuous time setting 
to make sure that the corresponding Markov chain was reversible and hence had a product-form 
distribution to start with; such reversibility is lost in general in the discrete time setting of 
this paper. Using this approximate product-form characterization of 7r(r) in addition to the 
Gibbs' maximal principle (cf. [6]), we prove that 7r(r) has the desired property; namely, that 

cji log log (5i(r) is close to vaaXp^x{G) J2i Pi ^^S ^og QiiT) if f = [cj] is given by the distribution 
7r(r). We call this the maximum- weight property at stationarity. 

In the second step, we show that the Markov chain, despite it being time-varying, is always 
near stationarity for large enough r by carefully estimating the effective mixing time of the 
time-varying Markov chain. In other words, the distribution of cr(T) is close to 7t{t) for large 
enough r. Therefore, the maximum-weight property at stationarity (established in the first 
step) implies that (Ti(r) log log (^^(t) is close to maxpg2:(c) />j log log (5i(T). To guarantee 
the near stationarity property as a consequence of such a mixing analysis, it is required that a 
design of 'weight' maintained by each node in the medium access algorithm utilizes the neighbor's 
weight information. As mentioned earlier, we resolve this by developing a learning mechanism 
that estimates the neighbor's weight based on the information whether it transmitted or not 
thus far. The success in this second step is primarily due to our novel design of the learning 
mechanism incorporated well with the mixing time analysis of the time-varying Markov chain. 

1.3 Organization 

Remainder of the paper is organized as follows. Section [2] presents formally the medium access 
algorithm and a statement of the main result. Section [3] summaries our proof strategy for the 
main result at a high level. Section d] presents necessary preliminaries that are used to establish 
the main result in Section O Section [6] and [7] are for providing the detailed proof of the main 
lemma in Section [5j 
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2 Algorithm and Its Optimality 



The medium access algorithm is randomized, distributed, simple and runs in discrete time with 
time indexed by r > 0. Each node i £ V maintains weight Wi{T) G R_|_ over r > 0. In the 
beginning of each time slot r > 0, each node i E V decides to attempt transmission or not as 
follows: 

1. If the transmission of node i was successful at r — 1, then 

o it attempts to transmit with probability 1 — 

2. Else if no neighbor of i attempted transmission at r — 1, then 

o it attempts to transmit with probability ^. 

3. Otherwise, it does not attempt to transmit with probability 1. 

Now we describe how each node i maintains weight Wj(r): 

Wi(r) = max jlog Qi(r), ^max exp (^^log g{A^{T))^ | , (1) 

where by log and log log we mean [log]_|_ and [log log] + respectively; g : M_|_ — )• is defined as 
g{x) = exp(log log^ x); by loglog^x we mean (log log j;)^, log represents logg; and J\f{i) = {j G 
V : G E} represents neighbors of node i. Note that Wi{T) > 1 for all r by definition. In 
above, is a 'counter' maintained by node i as a 'long term' estimate of weight Wj{-). This 

is maintained along with another 'counter' -Bj(-) by node i as a 'short term' estimate of Wj{-). 
Initially, A^{0) = B'j{0) = for all j G and i G V. For each j G Af{i), A* (•) and Bj{-) are 
updated by node i at r as follows: 



1. If J G attempted transmission at r — 1, then 

o A){t) = A)(t - 1) and B]{t) = B]{t - 1) + 1. 

2. Else if Bi(T- 1) > 2, then 

/ N f^t(^ - 1) + 1 if BUt - 1) > giA^Ar - 1)) 
oAUt) = 1 ^.^ ' '-^^ and resets 

|^A*(t — 1) — 1 otherwise 

3. Otherwise, A'-{t) = A^ir - 1) and ^^(t) = 0. 



Observe that B'j{-) counts how long neighbor j keeps attempting transmission consecutively. 
When j's transmissions are successful, the random period of consecutive transmissions is essen- 
tially distributed as per the geometric distribution with mean Wj{-) due to the nature of our 
algorithm. Thus Bj{-) provides a short-term (or instantaneous) estimation of Wj{-). To extract 
a robust estimation of Wj{-) from such short-term estimates, the long-term estimation A^j{-) is 
maintained: it changes by ±1 using Bj{-) at most once per unit time. Specifically, as per the 
above updates, g{AU-)) is acting as an estimation of Wj{-). Now it is important to note that the 
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choice of g (defined above) plays a crucial role in the quality of estimate of Wj{-). The change in 
estimation g{A^j{-)), when ^j(-) is updated by ±1, is roughly g'{A'j{-)). Since Wj{-) is changing 
over time, it is important to have g'{-) not too small. On the other hand, if it is too large then it 
is too sensitive and could be noisy just like A priori it is not clear if there exists a choice 

of any function g that allows for keeping ^j(-) for a good enough estimation of W"j(-), which sub- 
sequently leads to positive-recurrence of the network Markov chain. Somewhat surprisingly (at 
least to us), we find that indeed such a g exists and is as defined above: g{x) = exp(log log^ x). 
As per our proof technique, g{x) = exp(loglog" x) works for any a > 2; however we shall stick 
to the choice of a = 4 in the paper. Section [3] provides the reasons on why such a choice of 
function g is necessary and sufficient. Now we state the main result of this paper. 

Theorem 1 The medium access algorithm as described above is optimal for any interference 
graph. 

3 Proof of Theorem [It An Overview 

This section provides an overview of the proof of Theorem [T] to explain the key challenges 
involved in establishing it as well as intuition behind the particular choice of function g. The 
goal in this section is not to provide precise arguments but only provide intuition so as to assist 
a reader in understanding the structure of the proof. The complete proof with all details is 
stated in Sections El M and El 

Theorem [H requires establishing positive recurrence of an appropriate Markov chain that 
describes the evolution of the network state under the medium access algorithm described. To 
that end, define 

X(T) = (Q(r),o-(r),a(T),A(T),B(T)) 

where Q(t) represents vector of queue-sizes; ^(t) G {0, 1}" denotes the vector of transmission 
attempts by nodes at time r; <T(r) G I{G) denotes the vector of resulting successful transmissions 
in time r (clearly, (t^t) < ^(t)); and A(t), B(t) G denote the vector of long-term and 

short-term estimations maintained at nodes as explained in Section [2l Then it follows that under 
medium access algorithm X{-) is a Markov chain. 

Now a generic method to establish positive-recurrence of a Markov chain involves estab- 
lishing that certain real-valued function over the state-space of the Markov chain is Lyapunov 
or Potential function for the Markov chain. Roughly speaking, this involves establishing that 
on average the value of this function decreases under the dynamics of the Markov chain if its 
value is high enough; Theorem |H states the precise conditions that need to be verified. With 
this eventual goal, we consider the following function that maps state x = (Q,cr,a,A, B) to 
non-negative real values as 

Lix) = Y,F{Qi)+ E {i^f + g'^-'HB^)), (2) 

where F{x) = /^^ log logy dy with log logy = [loglogy]+; the inverse function of g{x) = 
exp(log log^ x) is 5f(~^)(x) = exp(exp(log^/^ x)). With abuse of notation, we shall use L{t) 
to denote L(X(r)). Now 

L(r) = L«(T)+L^'^(r), 
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where LQ{t) = T.^F{Q^{T)) and L^.^(r) = E,,eAr« {{A]{T)f + g^~^\B]{T))) . 

The proof is devoted to estabhsh the negative- drift property of L{-), i.e. if X{t) = x is such 
that L{t) is large enough (i.e. larger than some finite constant), then value of L(-) decreases 
enough on average. This property is established by considering two separate cases. 

Case One. When L(t) is large due to the component L^'^{t) being very large. 

o Formally, when max^j {g{A'j{T)), B'j{t)) > WI^^^{t), where VFmax(T) = maxiWiir). 
Case Two. When L(t) is large due to the component L^{t) being very large. 

o Formally, when max [g{A'j{T)), B^j{t)) < WI^^^{t), where VFmax(T) = maxj Wi(T). 

Case One. In this case, there exists i £ V and j G AA(z) so that g{Aj{T)) or Bj{t) is larger 
than Vl^max(''")- Using the property of the estimation procedure (which updates Aj{-)), we show 
that the L^'^{-) decreases on average by a large amount; it is large enough so that it dominates 
the possible increase in any other components of L{-). Such a strong property holds because as 
per the algorithm, g{A'j{T)) and -B*(r) continually try to estimate Wj{T) and hence if either of 
them is larger than W^^^{t), they ought to decrease by a large amount in a short time. Indeed, 
to translate this property into sufficient decrease of L{-), the careful choice of L^'^{-) is made. 
This case is dealt in detail in Section [6l 

Case Two. In this case, for each i £ V and j E AA(z), g{Aj(T)) and B^-{t) are smaller than 
W^s.^{t). To establish the decrease in -L(-), we show that in this case L^{-) decreases by large 
enough amount; large enough so that it dominates the possible increase in L'^'^(-). This case 
crucially utilizes the property of the medium access algorithm, the choice of the weights Wj(-) 
for i £ V and the form of function g. The precise details explaining how these play roles in 
establishing this decrease in L^{-) is explained in Section [71 Here, we shall provide key ideas 
behind these somewhat involved arguments. 

The property that L^{-) decreases by large enough amount follows if we establish that the 
set of transmitting nodes <T(r) is such that 

V CTj (r) log log gi(r) « max V pi log log Qi(T). (3) 

See Lemma [12] for further details. To establish ([3]), using the condition of the second case 
g(^*(r)) < VFj^ax(''") fo'^ alH G y and j G AA(i), we essentially show that 

g{,A){T)) ^ Wj{T), for all i e V, j e Af{i), and (4) 
(Ti (r ) log log Qiir) ^ max pi log Wi (r) (5) 

To see why ([4]) and ([5]) are sufficient to imply ([3]), note that 

I log Wj(r) - loglogQi(r)| < max Jlog5'(^Ur)) max A/logWj(T) 

< max pilogWiir), 
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when Wma.x{T) (or Qma.x{r)) is very large. Therefore, 

max y] Pi log log Qi (r ) f» max pi log VFj (r) . 

I % 

In summary, to establish desired decrease in L^{-), it boils down to establishing ([U and 

To establish ([4]), it is essential for g{-) to be growing fast enough so that if (r)) is very 
different (in this case, smaller) compared to VFj(r), then under the execution of the algorithm, 
it quickly converges (close) to Wj{-). For this, it is important that g{A^j{-)) should change at 
a faster rate compared to the rate at which Wj{-) changes. Towards that, note that if ^j(') is 
updated (by unit amount) then g(^* (-)) roughly changes by amount (jr'(A* (r)), which is at least 

g'iA)ir))>g'ig^-'\W^Urf)). 

Here we have used the fact that g' is a decreasing function and g{A^j{T)) is at most VFmax('7")^- 
Using properties of function g, we establish that (see Proposition [21]) Wj (t) changes per unit 
time by at most 

(exp {log^ Wj{t)))' 

For the purpose of developing an intuition regarding the choice of g, consider j G argmaxj Wi{T), 
i.e. Wj{T) = Wma.x{T). Then, such a Wj^t) changes as 

Wj{t) ^ VFn,ax(T) 

(-1) (exp (log2 Wj (r)) ) g(-^) (exp (log^ Wmax(T)) ) ' 
Therefore, to have g such that the change in Wj{-) is slower than that in g{A'j{-)), we must have 

gi 1) (exp (log^ Vrmax(T))) 

Our interest will be having properties holding when Wma.x{T) (or Qmax(T)) is large enough. 
This leads to the condition that 

// (-1)/ SNNf^'^U^^P^og^^)) 
lim g {g^ '{x )) — > 1. 

X— >-CX3 X 

It can be checked that the above condition is satisfied if g{x) does not grow slower than 
exp(log log" x) for some constant a > 2JI That is, we need g to be growing roughly at least as 
fast as the choice made in the description of our algorithm in Section [2j Precise details on how 
this choice of g guarantees ^(A* (r)) W"j(r) is given in Section [71 

Next, discussion on how we establish ([5]), which will require another condition on g{-) to 
be growing slow enough, in contrast to the fast enough growing condition for Q. Effectively, 
we need to establish that /i(t), the distribution of <T(r) under the algorithm, is concentrated 
around the subset of schedules with high-weight, i.e. roughly speaking the subset 

{~p=[pi]eI{G):Y,Pi^ogW,{r)^ max V;oaogI^,(T)|. (6) 
^—^ p€l(G) ^—^ J 

^We say g does not grow slower and faster than / if Vmix~^oo yrfy > and Yirax^x, jrfy < 00, respectively. 
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To that end, consider the evolution of schedule <t(t) = [o"j(T)] and weight W{t) = [Wi{T)] 
under the algorithm. Now the distribution of cr(r) depends on the schedule cr(r — 1) and weight 
W{t — 1). More specifically, the evolution of <t(t) can be thought of as a time- varying Markov 
chain with its transition matrix P{t) being function of the time-varying W{t). That is, for 
A > 1 

fxir)= /i(T-A)P(r-A)...P(T-l). 

In above, we assume that the distribution /x(-) represents an dimensional row vector, P(-) 

represents an x |X(G)| probability transition matrix, and their product on the right hand 

side should be treated as the usual vector-matrix multiplication. The first step towards estab- 
lishing concentration of /j(r) around the subset of I{G) with high- weight (cf. ([6])) is establishing 
the existence of an appropriate A > 1: 

(a) A is small enough so that 

P{t-A)---P{t-1)^P{t)^. 

(b) A is large enough so that 

/.(r- A)P(r)^p.7r(r), 
where 77(7) is the stationary distribution of P{t), i.e. 7v{t) = 7v{t)P{t). 

By finding such A, it essentially follows that h{t) ~ 7f(r). The second step towards establishing 
concentration of /x(r) around the high-weight set involves establishing that 7v{t) is approximately 
product-form with respect to the weights W{t) (cf. Lemma [2|). Therefore, as a consequence 
of Gibb's maximal principle for product-form distributions, it follows that 7t{t) is concentrated 
around the subset of I{G) with high- weight (cf. ([6])). Formally, this is stated in Proposition 
[TBI Subsequently, this establishes that /x(r) is concentrated around the subset of I{G) with 
high- weight (cf. ([6])) 

Now we discuss the remaining task of showing the existence of A so that (a) and (b) are 
satisfied. This is where we shall discover another sets of conditions on g that it must be of the 
form exp(log log" x) with a > 2. Now for (b) to hold, it is required that A is larger than the 
mixing time of P{t). Using Cheeger's inequality [U [21], we prove that it is sufficient to have 

A > /l(W„,ax(T)) With /i(x) = e(x6"+l). 

The precise definition of /i(-) is presented in Lemma [sH Next, for A to satisfy (a), observe that 

A) • • • P(r - s - l)(P(r - s) - P(r))P(r)^-i | 

s)-Pir)\\, 

where we use the triangle inequality with an appropriately defined norm || • ||. Further, by 
exploring algebraic properties of P(-) and W{-) (cf. Proposition 1151 and I2ip . we show that 

\\P{t - S) - P{t)\\ < /2(W^min(r)) • S, 
^As noted in Section [4.31 we use the asymptotic notation O with respect to scaling in VFmax(-) instead of n. 



\P(r-A)...P{T-l)-P{T)^\\ < Y.\\P{n 



=1 



< Eii^(- 

s=l 
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where Wmm{'^) = niin^ Wi{T) and f2{x) = @ "T^TTT — Ti — • Thus, it foUows that 

||P(r-A)...P(T-l)-P(T)^|| < f2{Wmin{r)) ■ 
Therefore, (a) follows if A satisfies 

A < — , for small enough e > 0. (8) 

From ([7|) and ([8]), it follows that a A > 1 satisfying (a) and (b) exists if 

/i(VFmax(T)) < / , ,„l == for large enough Qmax(T). (9) 

\//2(Wmin(T)) 

From ([1]), it follows that for any i £ V, 

Wi{T) > max exp ( Jlog5((A*(r))) « max exp I Jlog Ty,(T) I > exp I Jlog iy,(r) ) , 

(10) 

for any j E A/'(i); here we have assumed (7(A*(t)) VFj(t). Now let G argmiuj Wj{T) and 
j* G argmaxj Vrj (T). Since G is connected, there exists a path connecting j^, and j* of length 
at most D where D < n — 1 is the diameter of G. Then by a repeated application of (llOp along 
this path joining j^, and j* starting with j^,, we obtain that 

Wmm(r)> exp flogV^o iy^ax(T)V (11) 



Therefore, the desired inequality ([9]) is satisfied for large Wma.x{T) if 

e 



/l(^^max(T)) < 

(exp (logi/2'' T^^^(r; 

This holds if 



limsup/i(a;)W/2 (exp (log^/^^ x) ) = 0. 



The above can be checked to hold if g does not grow faster than exp(log log" x) for some constant 
a < oo. 



4 Preliminaries: Primary MC and Positive Recurrence 
4.1 A Markov Chain (MC) of Interest 

We describe a Markov chain of finite state space, whose time-varying version will describe the 
evolution of the medium access algorithm described in Section [2l As we described in Section [3l 
our strategy for proving Theorem [T] crucially relies on understanding the stationary distribution 
and mixing time of the (finite state) Markov chain. 
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Description. The Markov cham evolves on state space X(G) x {0, 1}" and uses node weights 
W = [W^] G M.I with Wrain > 1- Given {a, a) G I{G) x {0,1}", the next (random) state 
(<t', a') G 1(G) X {0, 1}" is obtained as fohows: 

1. Each node i chooses G {0, 1} uniformly at random, i.e. rj = 1 with probability 1/2 and 
otherwise. Temporarily set 




Tj if Gj = for all j G 
otherwise 



2. Each node i sets a'^ (and possibly resets a'^) as follows: 
o If (Tj = 1, then set 

, 1(0,0) with probability TFT- , 

1(1,1) otherwise. 

o Else if Uj = for all j G then set 

, [ 1 if a' = 1 and a' = for all j G M{i) 
I otherwise 

o Otherwise, set {a^,a'^) = (0,0). 

Stationary distribution. Let il. = I{G) x {0,1}'^. Then is the state space of the above 
described Markov chain; let Pxx' denote its transition probability for x = (<t, a), x' = (cr', a') G 
0. We characterize the stationary distribution of this Markov chain as follows. 

Lemma 2 Staring from initial state (0,0), the Markov chain P is recurrent and aperiodic; let 
its recurrence class be denoted by il.' C ^l; (c,0) G il.' for all cr G I{G). Therefore, the Markov 
chain P has a unique stationary distribution tt on 0' such that for any {cr,a) G 17' 

■^(cr,a) ocexp{(T -logW + U{(T,a)), (12) 

where U : ft' ^ M+ is such that \U{(T,a)\ < n4"log2 for all {a, a) G 0'. 

To achieve the form (|12p , we use the classical Markov chain tree theorem [3] . Our proof strategy 
can be of broad interest to characterize such form for non-reversible Markov chains. The proof 
of Lemma [2] is presented in Appendix lA.il 

Mixing time. Now we establish a bound on the 'mixing time' of P - the time to reach near 
stationary distribution starting from any initial distribution. We shall use the total-variation 
distance: given distributions u,fj, on a finite state space fi', define — /^HTy = J2xen' l^x — ^x|- 



Lemma 3 Given e G (0, 0.5) with n > 2, for any distribution /i on ^l' , 

ll/iP^ — 7v\\^y < e, 

for all T > Tmixie, n, "H^max), where 

Trmx = Tr^U^, U, W^^,) = 4"^"^ +'W^Z. log ( . (13) 

We use the Cheeger's inequality [11[21] to achieve the mixing bound ([13]). The proof of Lemma 
[3] is presented in Appendix IA.2[ 
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4.2 Ergodicity, Positive recurrence and Lyapunov-Foster 

To establish optimality of the medium access algorithm, we need to show that the underlying 
network Markov chain, which has countably infinite state space, is ergodic, i.e. that it has the 
unique stationary distribution to which it converges. We briefly recall known methods from 
literature that will be helpful in doing so. 

Consider a discrete time Markov chain X{-) on countably infinite state space X. State x G X is 
said to be recurrent if P(rx = oo) = 0, where = infjr > 1 : X{t) = x : X{0) = x}. Specifically, 
a recurrent state x is called positive recurrent E[Tx] < oo, or else if E[rx] = oo then it is called null 
recurrent. For an irreducible Markov chain, if one of its state is positive recurrent, the so are all; 
we call such a Markov chain positive recurrent. An irreducible, aperiodic and positive recurrent 
Markov chain is known to be ergodic: it has unique stationary and starting from any initial 
distribution, it converges (in distribution) to stationary distribution. Therefore, it is sufficient to 
establish positive recurrence property for establishing ergodicity of the Markov chain in addition 
to verifying irreducibility and aperiodicity properties. We shall recall a sufficient condition for 
establishing positive recurrence, known as the Lyapunov and Foster's criteria. 

Lyapunov and Foster's criteria. This criteria utilizes existence of a "Lyapunov", "Poten- 
tial" or "Energy" function of the state under evolution of the Markov chain. Specifically, consider 
a non-negative valued function L : X — t- such that supxgx-^(^) = oo. Let /i : X — t- Z+ be 
another function that is to be interpreted as a state dependent "stopping time". The 'drift' in 
Lyapunov function L in /i-steps starting from x G X is defined as 

E[L{X{h{x))) - L(X(0)) I X{0) = x]. 

Following is the criteria (see [5], for example): 

Theorem 4 For any k > 0, let Bf^ = {x : L{x) < k,}. Suppose there exist functions h,k : X ^ 
Z_|_ such that for any x G X, 

E [L{X{h{x))) - L{X{0)) I X{0) = x] < -k{x), 

that satisfy the following conditions: 

(LI) infxgx^(x) > — oo. 

(L2) liminfi(x)^oofc(x) > 0. 

(L3) sup^(x)<7 ^(x) < ^ /^'^ 7 > 0. 

(L4) limsupi(x)_^oo^W/^M < oo- 

Then, there exists constant kq > so that for all < ^> the following holds: 

E[Tb^ \ X{0) =x] < oo, foranyxeX 
sup E[rB^ I X{0) = x] < oo, 

where Tb^ ■= inf{r > 1 : X{t) G i?^} i-G. the first return time to B,^. In other words, Bf^ is 
positive recurrent. 

Theorem m implies that if (LI) - (L4) are satisfied and is a finite set, the Markov chain is 
positive recurrent. 
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4.3 Notations 



Let Z and M (Z+ and M+) denote sets of (non-negative) integers and reals, respectively. Bold 
letters are reserved for vector and distribution, e.g. u = [ui] denotes a vector; and 1 represent 
vectors of all O's and I's; for a function / : M — )• M, we use /(u) to denote [f{ui)]. Similarly 
for a random vector u, we use E[u] to denote Let Umax := maxjUj, Umin := miujUj. for 

a vector u; and u • v denote the inner product UiVi of vectors u, v. We call / : M — )• M as 
(uniformly) c-Lipschitz if \ f{x) — f{y)\ < c\x — y\ for some constant c > 0. Similarly, a sequence 
of random variables {A{t) G M : r € Z+j is c-Lipschitz if \A{t) — A{t + 1)| < c with probability 
1, for all r € Z+, for some constant c > 0. 

We will use asymptotic notations (e.g. 0,o,Cl,uj,@) with respect to scaling in queue-sizes, 
instead of the network size or something else. For example, we mean n = 0(1) and 2"Qmax = 
O(Qmax) where n is the number of nodes (or queues). We say function / : M_|_ — ?• ]R_|_ is 
polynomial by denoting f{x) = poly(x) if lima;_s.oo = for some constant c > 0. Sim- 
ilarly, f(x) = superpoly(3;) and f{x) = superpolylog(x) mean lima;_j.oo = oo and 
lima:^oo = oo for any constant c > 0, respectively. 



5 Proof of Theorem [T] 

We shall establish ergodicity of an appropriate Markov chain describing evolution of the network 
under medium access algorithm as long as A G A. To that end, recall the Markov state of the 
network defined in the Section [3) 

X(T) = (Q(T),a(r),a(r),A(T),B(T)), 

where recall that Q(r) represents vector of queue-sizes; a(r) G {0, 1}"" denotes the vector of 
transmission attempts by nodes at time r; cr(r) G I{G) denotes the vector of resulting successful 

'2\E\ 

transmissions in time r (clearly, <T(r) < a(T)); and A{t), B(r) G Z_,^ denoting the vector of 
long-term and short-term estimations maintained at nodes of the weights of their neighbors as 
explained in Section [2j Then it follows that under medium access algorithm X{-) is a Markov 
chain. It can be easily checked that under this Markov chain, state in which all components 
are 0, has positive probability of transiting to itself. Further, starting from any state, X{-) has 
positive probability of reaching state 0. Therefore, X{-) is always restricted to the recurrence 
class containing state 0; and over this class it is aperiodic. Therefore, it is sufficient to establish 
positive recurrence of X(-) over this recurrence class. Towards this, we shall utilize the following 
Lyapunov function L and auxiliary functions h,k to verify the conditions of Theorem [4l Given 
state X = (Q, a, a, A, B), define 

L(x) = J;F(Q,)+ Y1 + (14) 

where F{x) = /g^loglogy dy with log logy = [loglogy]+; let g^~^\x) = exp(exp(log^/^ x)) 
represent the inverse function of g{x) = exp(log log^ x). With an abuse of notation, we shall use 
L{t) to denote L(X(t)). 

Recall that node weights W are determined by Q and A as per ([T]). Therefore, given state 
X = (Q, <T, a. A, B), the weight vector W is determined. With this in mind, let 

C(x) = max{5((Amax), Bmax}- (15) 
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Then h and k are defined as 



C(x) 

^ exp (exp(log 



if C{x) > W: 



3 



h{x) 



max' 



(16) 



^ ^max)) otherwise. 



k{x) 



logl/2 W, 
2 



max 




if C(x) > Wl 
^ ^max)) otherwise. 



max' 



(17) 



With these definitions, we shall establish the following. 
Lemma 5 Let A G A. Then for any x with L{x) large enough, 

E[L(/i(x)) - L(0) I X(0) = x] < -A;(x). 



(18) 



It can be easily checked that L,h and k along with Lemma [5] satisfy conditions of Theorem [H 
Now L{x) — )• cxD as |x| — )• oo where |x| = |Q| + \cr\ + \a\ + \ A\ + |B| with \a\, \a\ being equal to the 
ordering of them and |Q|, \ A\ and |B| are standard 1-norm. Therefore, B,^ = {x : L{x) < k} is a 
finite set. Therefore, it follows that the Markov chain X{-) is positive recurrent; it is aperiodic 
and irreducible on the recurrence class containing as discussed before. Therefore, it follows 
that it is ergodic. That is medium access algorithm of interest is optimal establishing Theorem 
[TJ In the remainder this section, we shall establish the key Lemma [5j As explained in Section 
El the proof is divided in two cases: (a) for x with C(x) > Vl^max ^^"^ (b) otherwise. 

The case (a) corresponds to the situation when at least one of the estimation g{A'j{-)), Bj{-) 
of Wj{-) some neighbor j £ AA(z) for some node i is quite large. Therefore, in this case, due to 
the nature of the algorithm, we show that there is a reduction in the Lyapunov function (part 
that depends on A(-), B(-)). This is argued in detail in Section [6l 

In case (b), on the other hand, all estimations are not too large. Therefore, effectively the 
algorithm acts as if weight of each node, say node i, is such that 



Given this, as long as the Wmax (equivalently Qmax) is large enough, weight of each node is 
large enough (as it can be shown to be lower bounded by some increasing function of Qmax)* 
Therefore, weight of each node changes very slowly : each component of Q(-) changes at most 
by unit per unit time and hence if Qmax is large then log Qmax changes by small amount per 
unit time. This essentially 'freezes' the weights over a time period that is long enough for the 
corresponding Markov chain of (<T(-),a(-)) to reach its stationary distribution (using bound on 
Mixing time cf. Lemma [3]). We show that the stationary distribution has property that (with 
respect to it) on average the first part of the Lyapunov function decreases maximally; it results 
into overall negative drift if A G A. This will be useful to conclude Lemma [5] in case (b). This 
is argued in detail in Section [71 



The goal is to establish that starting with state X{0) = x = (Q,cr,a, A, B) such that L{x) is 
large enough (to be determined in the course of the proof) and C(x) = max{5((Amax)5 Bmax} 
> VFj^j^x (with W determined based on Q, A as per ([T])), after time h(x) = C(x)" the expected 
value of L decreases by A;(x) = /i(x)^ = C(x)^". 




6 Proof of Lemma St C(x) > Wl 



max 



E[L{h{x)) - L(0) I X{0) = x] < -k{x). 



(19) 
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We note that for proving Lemma [5l these will be the definition of functions h and k as it 
concerns the case C(x) > Vl^max- To simplify notations, we will use notation E[-] and P[-] 
instead of E[- 1 X{0) = x] and P[- 1 X{0) = x] whenever it is clear from the context. 

To that end, note that if L(x) is large enough, then either Qmaxi ^max or Bmax is large. Now 
if Q max or Ajnax stve large then necessarily VFmax is large. Since C(x) depends on Amax) Bmax 
and since we have C(x) > W^^^, it necessarily follows that C(x) is large due to L(x) being 
large. Now for large enough L(x) and hence large enough C(x), 



ff(-i) exp(log2(yCW72-l)) 
h{x) = C(x)" < 



2(7CW72-1) 

The above holds for large enough C(x) because (7^"^^ (exp(log^ x)) is a super-polynomial function 
of X, i.e. 

5f(~i)(exp(log2x)) J 
> 00, as X — )• 00, tor any hxed c > 0. 

x^ 

Therefore, from (j94p of Corollary [22] (presented in Appendix), it follows that for r < h{x), 

A 



^^max(T) < VC{x)/2 = Wmax. (20) 



Two Lemmas. Now we state two key lemmas that will lead to ()19p . We shall present their 
proofs in Section [6.11 and [6.21 respectively. 

Lemma 6 Given initial state X{0) = x = (Q, <t, a. A, B), let C(x) > Vt^^ax "-''^^ ^'(x) be large 
enough. Then for any i and j £ AA(f) 

E[A]{h{x)f] < I o(3(Ap^j (21) 

\^{Aj + h{x))'^ otherwise. 

Lemma 7 Given initial state X{0) = x = (Q, cr, a. A, B), let C(x) > Vt^^ax ^(^) large 
enough. Then for any i and j S 

E[g^~'\B){h{x)))] < 0[g(-'\C{x)/2)). (22) 
Implications of Lemmas [6] and [3 Define five different events as follows: 



Si 


= {(z 


j)^E 


g{A]) = C(x)} 


^2 


= {(i 


j)^E 


C7(x)/2 < g{A]) < C{x)} 


^3 


= {{i 


j)^E 


g{A]) < C{x)/2} 


^4 


= {{i 


j)^E 


B] = C(x)} 


^5 


= {{i 


j)^E 


B] < C(x)}. 
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For E ^i, using relation g{A^j) = C(x), h{x) = C(x)" and Lemma [6l we have 
E[4(Mx))2 - {A)f] < /^'^^l,, + 0{A]) 

o(<7(a;.)^^ 



0(c(x) + ) ^ 



<-i5(-i)(C(x)), (23) 

where the last inequality follows for large enough C(x). 

For G 5*2, it follows from Lemma [6] that for large enough value of C(x) 

E[A]{hix)f - iA]f] <0, (24) 



n + l 



where we use g{A))— = o(C7(x)") and g{A)) = n{C{x)). 
For € iSs, Lemma [6] implies that 

E[4(/i(x))2-(yi;f] < (a;. + /i(x))2 - (4)2 = 24. /i(x) + /i(x)2 

< 2,(-i)(^)Mx) + Mx)^ = 0(5(-^)(^))W, (25) 

where the last inequality utilizes the super-polynomial property of g^~^\-) function: 

Mx) = C(x)" = o(5(-i)(^)). (26) 

For S 5*4, Lemma [7] implies that 

E [,(-1) (B^ (Mx))) - ,(-^) (i?i)] < O (,(-^) (^) ) - ^^^"'^ (^W) 

<-^5(-'^(^^(x)), (27) 

where the last equality follows for C(x) large enough from the following proposition stating the 
super-polynomial property of g^~^\-) function; we skip the proof as it is elementary. 



Proposition 8 For any given k G 



hm ^ > = 0. 28 



For (i,j) € 55, Lemma [7] implies that 

'C(x) 



^[g^-'){B]{h{.)))]=0U~^^i^)). (29) 
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Prom (123|), (IMI), (ESj), ([27]) and (1^ follows that 

< -{^^^^)9^~'HC{^)) + {\S3\ + \S,\)0{g(~^^{^))c{.r 

where (a) is from \Si\ + |5'4| > 1 and l-Ssj + jS'sj < \E\ <r?\ (b) is from Proposition El 
Concluding (I19p . Prom the above inequality, we obtain the following: 

E 



L(/i(x)) - L(0) 

= E [ F(Q,(Mx))) + 4(^W)' + E 5^"'^ (^j(M>^))) 
- E [E m) + + E ^^"'H^;) 

<E[Ei^(ft(Mx))) - - ^5("'HC(x)) 

i 

< E (^(ft + Mx)) - i^(gi)) - ^5^-^^ (c(x)) 

i 

<E/(^^ + Mx))Mx)-^5(~')(C^(x)) 

i 

< n/(Q,,ax + /i(x))/i(x) - ig(-i)(C(x)) 

< n/(exp(iy„ax) + /i(x))/i(x) - ^^^"'^^(x)) 



< n/(exp(C(x)i/3) + C7(x)")C(x)" - ^^("^^(^(x)) 



where (c) is from 1-Lipschitz property of Qi{-); (d) is from C(x) > VF^^x! (s) ™d (f) hold for 
large enough C(x) due to the fact that f{x) = log log x and g^"^\x) has the super-polynomial 
growth property as per Proposition [HI This completes the proof of Lemma [5] for the case 

^^(x) > wLx- 

6.1 Proof of Lemma [6] 

Observe that Lemma [6] for the case g{A^j) < C(x)/2 follows immediately from the 1-Lipschitz 
property of AU-). Hence, we shall only consider the case when ^(AM > C(x)/2. 
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Define Wmax = \/C(x)/2. Then from ^ 

Wmax(r) < Wmax < ^fii^) , for all r < /i(x). (30) 

For the purpose of the proof, consider modification of the original network Markov chain, say 
original Markov chain be M and let its modification be M'. Under this modification, the M' 
evolves in the same manner as M for r < h{x); for r > h{x) the evolution of M' is the same as 
that of M except W{t) = W{t — 1) i.e. W{-) remains fixed to its value at time h(x). Clearly, 
the quantity of interest E[j4* (/i(x))^] in Lemma [6] remains invariant with respect to M and M'. 
As mentioned earlier, this modification is for convenience of proof and it merely guarantees (j30p 
for all T. Therefore, we shall bound E[A* (/i(x))^] under M' for which we have 



Vl^max(r) < Wmax < ^J9iA)), for ah r > 0. (31) 

With respect to M' , define random times = Tq < Ti < T2 . . . such that is the mth time 
when A'jl) is updated, i.e. Bj{Tm - 1) > 2 and B'j{Tm.) = 0. Define for m > 0, 

= 1 if ^ or m = ^^^^ 

]Ym^i — Aj otherwise. 

Let m* = inf{m > : > h{x)}. Then it follows that 

A){h{x)f = A'^iTrrr*-!? = (33) 
We establish the fohowing property of Ym- 
Proposition 9 Given g{Aj) > C(x)/2, for m> 1 

E[Ym+l I Tm]<Ym-A'j, 

where Tm denotes the filtration containing Yk,Tk for < k < m. 

Proof. If Tm > h{x), then the desired result follows from definition of Y^. Now suppose 
Tm < h(x). Then observe that 

5(4(r™)) > 9(4 -h{x)) 

(b) . . . 

> giA'j) - h{x)g'{c), for some c G {A'j - /i(x), A*) 
>g{A^^)-h{x)g'{A]-h{x)) 

>g{A))-l, (34) 

where (a) is from 1-Lipschitz property of A'j{-); (b) is from the mean value theorem; for (c) we 
use the fohowing that holds for large enough C(x) and g{Aj) > C(x)/2 (along with the definition 
of5(-)) 

h{x) = C7(x)'^ < 2-'^5(4.)" < 
and g'{x) < for large enough x. 
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Now we bound the probability that A) increases at time Tm+i- 



¥[A){T^+i) = A]{Tm) + l 



J~ rr 



l)>g{AUT^)) 



<F[B}iTm+i-l)>giA])-l 

1 \9(4)-2 



^ m. 



1 



{d) 1 

< — . 

- 10 



(35) 



In above, (a) and (c) are from (j34p and (|3ip when assuming A*- (equivalently C(x)) is large 
enough. For (b), we observe that Wraa^xiT) is uniformly bounded above by Wmax from (f3T]l . 
Therefore, once j is successful in its transmission, the probability that j consecutively attempts 



to transmit (without stopping) for an interval of length k is at most |^1 — — 
it follows that 



Using ([35 



E 



m+l 



E 



A^j(Tm+l 



10 

8 

Aj(Tfn) — —Aj[TjYi) 



<A]{Tmf 

<A]{T^f-'±A] + "-h{^) 



where we used 1-Lipschitz property of A^j{-), < h{x) and the fact that h{x) = C^xy- < 
2~"'g{A'j)"' = o{A'j). This completes the proof of Proposition [9l □ 

Completing proof of Lemma [6l Define Zm = Ym + {m — l)^j- Then as per Proposition 
[HI {^rn : m > 1} is a sub-martingale with respect to J-m- By the Doob's optional stopping 
theorem, we have that 

E[Z„,] <E[Zi] =E[yi]. 



Therefore, the desired inequality follows as 



E 



M{hix)f 



AiiT^^-i) 

A){T,r,,f 



(a) 

< E 



{A'^{Trn^) + lf 



+ 2E 



A){T^*] 



+ 1 
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(b) 

< E 



= E 
< E 
= E 



Y . 
Yi 



+ 2E A'j + m* 
- {m* - l)A] 



+ 1 
+ 2E 



A] + m* 



+ 1 



E 



m 



[A] - 2) + + 1 



E 



m 





{A]f 



■E 



[A] - 2) + 3^;. + 1 
A] - 2) + 34 + 1 



E 



m 



[A] - 2) + 5^;- + 2 



/i(x) 



0(g{A))- 



where (a), (b), (c) are from the 1-Lipschitz property of A^-{-) and (d) is due to the following 
proposition. This completes the proof of Lemma [H 



Proposition 10 For large enough C{x), 

E\m*] > 



hix) 



O UA) 



n + l 
2 



+ 1. 



Proof. For 1 < r < /i(x), define 



1 if AU-) is updated at time r 



I otherwise. 

That is, [/^ = 1 iff 5Ut - 1) > 2 and B\{t) = 0. By definition of Ur and m*. 



m 



hix) 

T = l 



Therefore, to bound E[m*] we next bound E[[/t-]. For any 5 < r < h{x) — 5, let X{t — 5) = 
{Q(r — 5), <T(r — 5), a(T — 5), A(t — 5), B(t — 5)} be the network state at time r — 5. For this, 
define event £: 

£ = ^2 & £3 

e;i = all nodes do not attempt to transmit at time r — 4 
£2 = Only j attemtps to trasmit at time r — 2 and r — 3 
6:3 = j does not attempt to transmit at time r — 1. 

If 2 happens, A^- is updated at time r i.e. C/r = 1- First note that 



n 



1 



(36) 
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whether this naive lower bound is obtained from (j3ip and the case when many nodes (as possible) 
succeed in their transmissions at time r — 4. Second we have 



1P[^2 I ^l] > 



(37) 



The above lower bound is obtained considering the scenario that the balanced coin of j produces 
'head' at times r — 2 and r — 3; coins of all other nodes produce 'tail' at times r — 2 and r — 3. 
Third since the transmission of j is successful at time r — 2, it is easy to see that 

1 



F[(B3 I Cs] > 



n 



from jSU. By combining ([36]) . (l371) and (p8]l . 

¥[Ur = 1\X{t- 5)] 
> P[e I X(r-5)] 
= P[ei & £2 & ^3 I X{t - 5)] 

The above inequality holds for any given X[t — 5). Hence, 

= 1] = n {g{Ai 

Finally, the conclusion follows as 



n + l 
2 



n + l 
2 



E[m* - 1] > E 



h{x) 
r=5 



h(x) 

T=5 

/i(x) 

T=5 

(Mx) -4) .0(5(4.)- 
/i(x) 



n + l 
2 



o 5(4. 



n + l 
2 



(38) 



□ 



6.2 Proof of Lemma [7] 

Let r* = inf{r > 1 : aj(T) = 0} i.e. the first time j does not attempt to transmit, and let the 
event £ denote r* > h{x). Hence, if £ happens, Bj{h(x)) = Bj + h(x) and transmissions of j 
should be successful consecutively for time r € [0, h(x) — 2] (otherwise, j would have stopped 
attempting). Under this observation, we obtain 



[<t] < P [j attempts to transmit consecutively for time r G [1, h{x) — 1 



1 \^(x)-l ^ ^ 



20 



where the last inequahty follows from the fact that VFj (t) is bounded from above by Wmax as 
per (I20p . On the other hand, if the event ^ does not happen, j stops attempting transmission 
before time h{x). Hence Bj should set to before time h{x). Based on this observation and 
arguments similar to those used for establishing (|39p . we obtain 

¥[B'j{h{x)) = k I ^'^] < ¥ j attempts to transmit consecutively 



for time r G [h{x) - k + 1, h{x) - 1] 

[ 



1 \ k-1 

< (1-^7^) ■ (40) 



Now observe that 



E 



g^~'HB}{h{x)))\ = ¥[m9^-'\B}{h{x))) \ (£] +¥[(£'^ng^-'\B'^{h{x))) \ 

< ¥[m9^~'HBi{h{x))) I ^]+E[g(~'\B}{h{x))) \ r]. (41) 

For the first term in ()4ip . we consider the following using ([39|): 

P[C:]Eb(-^)(i?;.(/.(x))) I ^]<(l--L_Y'^^^~\gi-^){B^^+h{x)) 

^ ' ''max ^ 

< (l-^^Y^^^"~'-9^~'\C{x) + C-{x)) 

= 0(1), (42) 

where the last inequality follows for C(x) large enough, i.e. W^max large enough. In above we 
have used the definition h{x) = C(x)" with C(x) = max{g{Aj^i^x), ^max} which is at least W^max- 
For the second term in (I4ip . we consider the following using (I40p : 



E[g(~'HB^{h(x))) I G:^ < J^^^-^^A:) • (l - -— ) " 

, -, \ y Vmax ^ 

fe=l 

o(,(-^)(wLx)) 

O (C(x)/2)) , (43) 

where (b) is from ()20p and for (a) we prove the following technical proposition whose proof is 
presented in Appendix lC.il 

Proposition 11 For p G (0,1), 

oo 

J2g(~^)(k).{l-p)'^ = 0[g<^-'^p- 

k=l 



Combining (j^T]) . ()12|) and (jl3|) . the desired conclusion of Lemma [7] follows. This completes the 
proof of Lemma [71 
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7 Proof of Lemma B C{x) < W'l^^^ 

We remind that the goal is to estabhsh that starting with state ^(0) = x = (Q, cr, a, A, B) such 
that L(x) is large enough and C(x) = max{g{Ay^s,x),^max} < ^max' after time 

h{x) = ^ exp (exp(log^/^ Vl^max)) , 

the expected value of L decreases by 

1 /2 

A:(x) = exp ( exp(logi/2 W^.^)) = log'/' W^.. h{x). 

In other words, 

E[L(/i(x)) - L(0) I X(0) = x] < -A;(x). (44) 



We note that for proving Lemma O these will be the definition of functions h and k (cf. (jl6p 
and (I17p ) it concerns the case C(x) < Vl^max- '^o simplify notations, we will use notation E[-] 
and P[-] instead of E[- 1 X{0) = x] and ¥[■ \ X{0) = x] whenever clear from the context. 

Similar to the proof of Lemma [5] for the case C (x) > H^^ax (presented in Section [6|) , we 
start by obtaining some bound for H^max(''")- To this end, note that if L(x) is large enough, then 
either Qmax) -'4-max or Bijiax are large. Since Qmaxj -^max 

and Bmax are bounded in terms of 
^max, ^max IS uecessarily large if L(x) is large enough. Now for large enough L{x) and hence 
large enough Vt^max, we have 

VTmax = max I log Qmax , exp ( y^log S'( Amax) ) } 
max I log Qmax, exp(v^31og T^max} 



(a) 

< 



= log Qr 



where (a) is from the condition 5f(Amax) < C'(x) < V^max and (b) is because Vl^max > 
exp (-^3 log W^max) for large enough W^max- Henceforth, we shall assume that 

VTmax = log Qmax (45) 

and consequently Qmax can be also assumed to be large enough if L(x) is large. Using this, we 
obtain the following lower bound of Wtna.x{T): for r < h{x), 

(a) 

Wmax(T) > logQ 

max (r) > log(Q max 

= log(Qmax - o(Qmax)) 
(c) 1 

> -log Qmax := Wmin, (46) 

where (a) is from 1-Lipschitz property of Qmax(')> (c) holds for large enough Qmax and (b) is 
due to 

= ;^ exp ( exp(log^/^ VTmax)) = exp ( exp(log log^/^ Qmax)) = o(Qmax)- 
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On the other hand, VFmax(T) can be upper bounded as follows: for r < h(x), 

VFmax(T) < max jlog Qmax(T), exp(Y/log g( Amax(T")) ) } 

id) ( 

< max |log(Qmax + /i(x)),exp(Vlog5r(Amax + h{x))) j 

(e) 

< max {log(Qmax + o(Qmax)), log Qmax} 

< 2 -log Qmax, (47) 

where (d) is from 1-Lipschitz properties of Qmax(')> ^max(0 and (e) follows from below using 

-'^max 

<ff(-i)(C(x)), C{x)<W 

max- 

Vlogff(Amax + /j(x)) < ^logg{gi~^){C{x)) + h{x)) 



< Jlogg{gi-^){wU+h{x)) 



log g{g(-^) (log3Qmax)+/i(x)) 



(/) 



< ^logg{2-h{x)) 



logg (^exp(exp(loglog^/^ Qmax)) 
= log log Qmax- 

In above, for (f) one can check 

(log^ Qmax) < h{x) = ^ exp(exp(log log^/^ Q^^^)) 

for large enough Qmax- Combining (jl6|) and (|T7|) . it follows that for r < h(x), 

Wmin < Wmax(T) < Wmax, (48) 
where Wmin := I log Qmax and Wmax := 2 log Qmax- 

Three Lemmas. Now we state the following key lemmas that will lead to (|44p. We shall 
present their proofs in Section 17.11 17.21 and 17.31 respectively. 

Lemma 12 Given initial state X{0) = x = (Q, <t, a, A, B), let C(x) < VF^^x; A G A and Qmax 
be large enough. Then 

E [^F(Q,(/l(x)))] <Y,HQ^) - f^(log log Qmax) " Hx). 

i i 

Lemma 13 Given initial state X{0) = x = (Q, cr, a. A, B), let C(x) < "H^max ^'^'^ Qmax be 
large enough. Then for any i and j € A/'(i) 

E[A^{h{x)f] = 0{h{x)). 
Lemma 14 Given initial state ^(0) = x = (Q, <t, a, A, B), let C(x) < Vl^max one? Qmax 

be 

large enough. Then for any i and j € A/'(i) 



E[g(-^\B}{hix)))] = o(<7(-i)(41og2Qmax)) 
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Concluding ()44p using Lemma ll2|, 1131 and 1141 These lemmas lead to the desired conclusion 
as follows: 

E[L(Mx))-L(0)] 

i i i,j 



< -J](loglogQ^ax) • h{x) + 0{h{x)) + O (g^-^\Alog^ Q^^x) 

(a) 

< -f](loglogQmax) • /l(x) 
= -nilogWra..) ■ h{x) 

(b) 

< -logl/2iy„,ax-/l(x), 

where (a) is because g^~^\4:log^ Qmax) = o{h(x)) for our choice of /i(x) = ^ exp ^exp 

(loglog^/2Q 

and (b) holds for large enough Wmax- This completes the proof of Lemma [5] for the case 

c{x) < wLx- 

7.1 Proof of Lemma 1121 

We start by observing that 

E[Y,F{Q,{h{y^)))-Y,HQ^) 

i i 
h{>C)-l 

= E E[j;F(g,(r + l))-^m(r))' 

r=0 i i 

h{x)-l 

= ^ ^E[F(Q,(r + l))-F(Q,(r))] 

r=0 i 

< Yl Y.^[mr+l)-Q^{T))■fmr+m 

h,(x)-l 
r=0 « 

where (a) is due to the convexity of F and the last inequality is from 1-Lipschitz property of 
Qi{-). For each term in the summation of (1490 . we consider the following. 

n{Qi{r + l)-Q^{T))■f{Q^{T))] 

= E [{Mr) - a,(r)I|Q^(,)>o}) • fmr))] 
E [A,{t) ■ fmr))] - E [<y^{T) ■ fmr))] 

E [A, • fmr))] - E ki(r) • /(Qi(r))] , (50) 
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where for (a) we use I{Q,(r)>o} " fiQii'^)) = fiQiij)) since /(O) = 0; for (b) we use the fact that 
^j(r), Qi(r) are independent random variables and E[74i(r)] = Aj. Now from (H9]) and ([50ll . it 
follows that 



E 



/i(x)-l 

r=0 i 
h(x)-l 

= E f{Qr{r))-Y.a,{T)- f{Q^{T))]+0{h{x)) 

T=0 i i 

At(x)-1 

< max p./(Q(T)))-«T(r)./(Q(T))]+0(/i(x)), 

where the last equality is from A = [Ai] G A C (1 — Conv(X(G)) for some e > and the convex 
hull Conv(X(G)) of Hence, for the proof of Lemma [121 it is enough to prove that 

/i(x)-l 

lE[(l-e)( max p./(Q(T))) -tT(T)./(Q(T))] =-J7(loglogQ^ax)-Mx). (51) 
Further, it suffices to prove that for some R = o{h{x)) 

h(x)-l 

J2 E[(l-e)( max p./(Q(T))) -^(t)./(Q(t))] =-J](loglogQ^ax)-(/i(x)-i2), (52) 

since the remaining terms in (j5ip . other than those present in (j52p . are dominated by (j52p as 
follows: 

5]E[(l-e)( max p./(Q(r))) -^(t)-/(Q(t))] 

_R-1 R-1 

< J;e[ max p./(Q(T))] < E [n • /(Q„a.(r))] 

< ^E[n-/(exp(iy„ax(r)))] = 0(i?) • / (exp (W^a.)) 

r=0 

= o(/l(x)) • loglogQinax, 

where the last equality is from /(x) = log log x, Wmax < 21ogQjnax (cf. (^8]) ) and R = o(/i(x)). 

Now we will proceed toward proving ()52p . Equivalently, we will find some R = o(/i(x)) such 
that for aU r G [i?, /i(x) - 1], 

E[(l-e)( max p • /(Q(r))) - cr(T) • /(Q(r))] = -0(log log Q„,ax). (53) 
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Three sub-Lemmas. The proof of (j53p will be established as a consequence of following three 
lemmas. Their proof are presented in Section [7.1.11 [7TL2] and [7.1.31 respectively. 

Lemma A Let h{t) denote the distribution of {cr{T),a(T)} at time r. Then, 

r-l 

\\Kr)-S{^,a}-P{Or\\Tv < 0[Y.E[\\P{s)-Pmu), 

s=0 

where Sx is the Dirac distribution of singleton support x and P{t) denotes the transition matrix 
for the Markov chain corresponding to the schedule o'(t), as described in Section [4.11 as well as 
Section [3] which is function of node weights W{t) determined by Q(r) and A(r) as per ([T]). 

Lemma B Let event £r at time r be as follows: 

£r ■■= {X(r) :t^,(r) >exp(loglog''Qmax) and 

5(4- (r)) < log^ Q„,ax for all i,j G M{i)}, 
where r] := 1/4". Then, there exists Rb = polylog(Qmax) such that for r < h{x), 
{l-e)( max p • /(Q(t))) - E [ct{t + Rb) ■ /(Q(r)) | X(r) G £r] 

< -| • loglogQmax- 



Lemma C There exists Rc = o(/i(x)) such that £r happens with high probability for r G 
[Rc,h{x)] i.e. 

¥[£r] = l-o(l), 

where recall o(l) means that o(l) — )■ as Qmax oo. The o(l) bound is uniform over all r. 

Remarks for Lemma A, B and C. Before we derive the desired inequality (j53p using the 
above lemmas, some remarks are their role in establishing it are in order. To start with, the 
Lemma A captures the evolution of the distribution of schedules. It is used crucially to establish 
Lemma B. Lemma B implies that (|53p holds at time t + Rb '\i £t happens at time r and Rb 
small enough to guarantee /(Q(t)) « /(Q(T + i?B)). Lemma C indeed suggests that such event 
£r happens with high probability. This will essentially lead to ()53p . 

Concluding ([53]) . We choose R for ([53]) as 

R = Rb + Rc- 

It is easy to check R = o(/i(x)) since Rb = polylog(Qjnax) = o(/j(x)) and Rc = o{h(x)). For 
T G [R, h(x) — 1], we break the left hand side of (f53|) into two parts as follows: 

E[(l-e)( max p./(Q(r))) -a(r)./(Q(r))] 

= F[£r^nn]-Ka-e)( max p • /(Q(t))) - «T(r) • /(Q(t)) \ £r-R,] 

+FK_nJ-E[{l-e)( max p./(Q(T))) -^(t)./(Q(t)) | £:,^„^ J . (54) 
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For the first term in (|54p . we obtain 

F[£r-R,]-E[il-e)( mnx p.f{Q{T)))-a{T).f{Q{T))\£r^R,] 
Vpgl(G) / 

(1 - 0(1)) • E [(1 - e) ( max p • /(Q(r))) - a{r) ■ /(Q(r)) | Sr^R,] 

< (1 - o(l)) • E [(1 - e) ( ^max^ p ■ f{Q{T - Rb)) 

-a{T) ■ f{Q{T - Rb)) \ Sr^R^] - 0{f{RB)) 

< -(1 - 0(1)) • I • loglogQ^ax - 0{f{RB)) 

id) e 

< -- • loglogQmax, (55) 

where (a) and (c) are from Lemma C and Lemma B, respectively. For (b), we use 1-Lipschitz 
property of Qi{-) and \ f{x) — f{y)\ < f{\x — y\) + 0(1) for f{x) = log log x. For (d), we use 

/{Rb) = / (poly log (Qmax)) = o(/(Qmax)) = o(log log Qmax) • 

For the second term in (1541). we observe that 



P[f,^_^J-E[(l-e)(^max^p-/(Q(r))j -tT(r)./(Q(r)) | S:;.^^] 

<o(l)-E[( max p./(Q(T))) \ £!;_j,^] 
<o(l)-E[n./(Q^ax(r)) | S^^.r^] 

= o{l)-0{fiQ,n..)) = o(loglogQ„,ax), (56) 
where (e) is from Lemma C and (f) is due to 

/(Qmax(T)) < /(Qmax + r) < /(Qmax + /i(x)) 

= /(Qmax + o(Qmax)) = 0(/(Qmax))- 

Finally, combining (f5l]l . (fSSl) and (f56l) . the desired (f53]l follows as 

E[(l-e)( max p-/(Q(r))) -,T(T)-/(Q(r))l < • log log Qmax- 

VpGX(G) / J 16 



This completes the proof of Lemma [T21 



7.1.1 Proof of Lemma A 



Let p{t + 1 : r) be the distribution of {<T(r + 1), a(r + 1)} at time r + 1 given network state 
X{t) at time r. From the definition of P{t), we have 

/l(r + 1 : r) = 5{o-(r),a(r)}^(-^), 
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where recall that P{t) is a function of the network state X{t) since node weights W{t) is decided 
by Q(r) and A{t) as per ([T]). By taking expectations on both sides of the above equation, we 
obtain 

/x(r + l) = E[S{^(^r)Mr)}Pir)], 

where the expectation is with respect to the distribution of X(t). Using the above relation, we 
have 

/i(r + l) = E [<5{,(,),„(,)|P(r)] 



E 
E 



^{<T(r),a{r)}^(r) Q(r),A(r) 



'{<T(r),a{r)} 



Q(r),A(r) • P(r) 



E[/i(r)-P(T)], 



where the expectation is with respect to the distribution of {Q(r),A(r)} and we have used 
notation 

/i(T) = /i(Q(r), A(r)) := E [S{^(^r)Mr)} \ Qi^), A{t) 
This leads to the following recursive relation between /x(t + 1) and h{t). 

fx{T + l) = E[/i(T).P(T)] 

= E [/i(T) • P(0)] + E [/i(T) • (P(t) - P(0))] 
= E[/i(T)]-P(0) + e(T) 
= /x(r).P(0) + e(T), 

where we define 

e(r) :=E[/i(r).(P(r)-P(0))]. 
By recursive application of this relation, we obtain 

T-l 

/x(r) = ;.(0).P(0r + ^e(T-l-s)-P(0r 

s=0 
r-1 



s=0 



Now we obtain the desired conclusion of Lemma A from the above inequality as follows: 
\Kr)-S{^,a.}-Pior\\Tv 



TV 



\TV 



-"1 

< ^||e(s).P(Ori- 

s=0 

< o(^xi^[||p(s)-p(o)|U]^ 
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where we have used the fact that P(O)'^ ^ (resp. fJ-^r)) is a transition matrix (resp. distribution 
vector) of finite dimension, independent of initial state x. This completes the proof of Lemma 
A. 

7.1.2 Proof of Lemma B 

Recall that r is time such that even £r holds, where 

£r := {x(t) :t^,(T) >exp(loglog^Q„,ax) and 
5(4(t)) < log^ Q„,ax for ah ij G 

with r] := 1/4". We wish to show the existence of Rb so that Rb = polylog(Qmax) and 

(l-e)( ma^ p-f{Q{T)))-K[a{T + RB)-f{Q{T))\X{T)e£r] < - ^ • log log Q^ax- 

To that end, we shall show that the above property holds for 

max ) 

n, 21og Q 

max 7 5 

where Tmix is defined as per as part of the statement of Lemma EJ Clearly, from definition 
Rb = polylog(Qmax)- Now given network state X{t) G £r at time r < h{x), we have 

||5Mr),a(r)}-^(T)^^-^(T)||^^ < ^ = o(l), 

where we let 7r(r) denote the unique stationary distribution of -P(r) and use Lemma [3] with 
^max(''") < Wmax = 2 log Qmax (cf. (jiH]) ). The above equality suggests the following: for 
distribution fi{T + Rb ■ t) oi a{T + Rb) given network state X{t), 



|/i(r + Rb:t)- 7r(r)||^^ 

< \\h{t + Rb:t)- 5{„i^^)^a(T)} ■ P{r 



Mb I 



\TV 



+ \\S{a{r),air)} ■ Pi^)^^ - ^(^)\\tV 
/t+Rb-1 \ 

= 0i E[||P(s)-P(r)|U] j +o(l) 

(a) \ 

<0 Yl K^ax\W^{s) - Wi{T)\] \ + o{l) 

"^V"' n( i^-r)-2exp (log log^ Q^ax) \ , ... 

" ^ V 5^-'^ (^^p 1°^'" Q-x)) J 

W polylog(Q 

max J 

2 exp (log log'' Q 

max 7 , /-I \ 

" (exp (loglog^^Q.^ax)) 

^=^(l) + o(l) = o(l), (57) 

where (a) is from Proposition [15] that is stated below (proof presented in Appendix 10.2^ : (b), 
(c) and (d) follow from the Corollary [22] in Appendix [B] by definition Rb = polylog(Qinax)5 
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Wi{T) > exp (log log'' Qmax) due to event £r and 

g("i) (exp (loglog^'' Q^ax)) 
exp (log log'' Q 

max 7 



superpolylog(Qj, 



Proposition 15 Given two weights = [Wl] and W"^ = [W^], let and he the Markov 
chains (i.e. their transition matrices) on Vt we described in Section \4.1\ using node weights 
and W"^ , respectively. Then, 

\pL' - Pxx' I = 0(max\wl -W,^\), for all x, x' G 0. 

i 

From the above inequality (|57p . it follows that 
E [a{T + Rb) ■ log W{t) I X{t) £ £r] 

>E [a^^^y log W{t) I X{T)e£r]-Mr + RB:T)-7v{T)\\^y( max p-logW(T)) 

^> {1 - W^iir + Rb : t) - n{T)\\Tv) ( max p • log W(t)) - 0(1) 

= (1 - o(l)) ( max p • log W{t)) - 0(1), (58) 

where f^(^) G ^{G) is the random variable drawn by 7r(r) and for (a) we use the following 
proposition and the product-form characterization of 7r(r) in Lemma [2] (proof can be found in 
Appendix) . 

Proposition 16 (Gibbs' Maximal Principle) Let T : — )• M and let M.(VC) he space of all 
distributions on 0. Define F : — t- M as 

F(/x) = E[T{x^)] + Her{h), 

where Xf^ £ Q in the random variable drawn by /x and HEuitJ') is the standard discrete entropy 
of fx. Then, F is uniquely maximized by the distribution v, where 

Vx = exp {T{x)) , for any x e 

where Z is the normalization constant (or partition function). Further, with respect to v, we 
have 

lE[r(.Xj.)] > maxT(x) - log|0|. 

Concluding Lemma B. We further bound the difference between f{Qi{T)) and log Wiij) as 
\f{Q,{T))-\ogW,{T)\ = /(Qi(T)) -max{/(Q,(T)), max Jlog<7(4(r))| 



- \/log5(^K^)) 

(a) 



< y log (log^ Qmax) 
o(log log Qmax) 

0(/(Qmax)), (59) 
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where (a) is because X{t) £ Sr- Hence, we have 

E[a{T + RB)-f{Q{T)) I X{T)€£r] 

E [cr(r + Rb) ■ log W{t) \ X{t) G S^] - o(/(Qmax)) 

> (1 - 0(1)) ( max p . log Wir)) - 0(1) - o(/(Q^ax)) 

V p&X{G) / 

> (1 - 0(1)) ( max p • /(Q(r))) - o(/(Q„,ax)) 

> (l-o(l))( max p./(Q(T)) 
where (b), (d) are from ([59|) . (c) is due to (f58|) and (e) follows from 

max P-/(Q(t)) > /(Qmax(T)) > /(Qmax-r) 

peX{G) 

> /(Qmax - /i(x)) = /(Qmax - o(Qmax)) = ^/(Qmax), (60) 

for large enough Qmax- Finally, we derive the desired conclusion of Lemma B as 
(1 - e) max p • /(Q(t)) - E [cr(r + Kb) ■ /(Q(t)) | X{t) G £r] 

<-(6-o(l))( max p./(Q(r))) <-'-■( max p./(Q(r)) 

V p&I{G) / 2 V p&X{G) 

< -| • /(Qmax(r)) <-|-/(Qmax), 

where the last inequality is from ([60]) i.e. /(Qmax(''")) > /(Qmax)/2. This completes the proof 
of Lemma B. 

7.1.3 Proof of Lemma C 

We wish to show that there exists Rc = o{hix)) so that for any r G [iic*, /i(x)], the event £r 
holds. For this, it is sufficient to establish that for any r G [-Rc", /i(x)], the following holds: 

P[5(4(r)) <log^Q^ax] = l-o(l), (61) 

P[W'^(T) > exp(loglog''Q„,ax)] = l-o(l), (62) 
with the 0(1) being uniform over the choice of r G [Rc^ ^(x)]- We introduce some notations first: 

Ci := exp ( loglog^/^ Qmax ) and Ck ■= exp (log^/'' Ck^i) for A; > 2 



% ■■= Y^g^-'HCi /20) . log-+3 Q^^^^. 
1=1 

It can be checked inductively that 

Ck = exp (log log^/^'' Qmax) and £„ = exp (log log'' Q^ 
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where recall that t] = 1/4"'. Next, we shall show that both (|6ip and (j62p hold for the definition 
of Rc as follows: 

n-l 

Rc ■■= Tn-l = Y.9^~'^ («^P (loglog'^^' Qmax) /2o) • log"+3 Q^,,. 
fe=l 

Observe that as per this definition, Rc = o(/i(x)). Now we establish that indeed (I6ip and ()62p 
hold. 

Proof of del]). Define Tq = < Ti < T2 < . . . so that is the mth time when (•) is 
updated i.e. B'j{Tm - 1) > 2 and B'j{Tm) = 0. Define m as 

in := inf {m : Bj{Tm — 1) > 5^(7) and m > l} , 

where 7 = 51^"^^ (log' Qmax) — 2. In addition, note that 

g{A]) < C(X) < WLx = log' Qmax 

< log' Qmax -3 = g(7 + 2)-3 < 5(7-1) 

for large enough Qmax since for large values |5''(-)| ^ 1. Thus, Aj < 7 — 1. 

Now if A}(r) > 7 + 2 = ff(-^)(log^ Q max)) one can check that Tff^ < r since there should be 
at least two updates before time r which make ^j(-) increase beyond x. Otherwise, ^j(-) should 
remain less than 7 + 1 under the algorithm until time r since A^j < 7 — 1 in the beginning. 
Therefore, we have 

P[5(4(t)) > log' Qmax] = m]ir)>g^-^\log''Q^,^)] 

(a) ^ 
k=l 

< [aj{s) = 1 foT s = k - 2, ... ,k - \g{x)] - 1] 

k=l 

(c) ^ / I \ r9(^')i-i / I \ [9(^01 -1 

" ^ V ~ Wmaxj - ^' V^" 2 log Qmax J 

\ ^max / \ ^max / 

where (a) is from the union bound; for (b) we utilize the fact m > 1; for (c) one can observe that 
under the algorithm the probability that some node j keeps attempting to transmit consecutively 

(without stopping) for some time interval of length y is at most (l — prr — V (d) is due to 

X = 5^"^^ (log' Qmax) — 2; (e) is from h{x) = o(Qmax)- This completes the proof of ([6T]) . 

Proof of (I62p . We shall utilize the following result crucially whose proof is presented in 
Appendix [DJ 
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Proposition 17 Consider i G V, j G M{i)Ll{i}, W > and network state X{t) = {Q(r), <T(r), a(T), A(t), B(r) 
at time r < h{x). Suppose that Qmax is large enough and 

Wiir) > W > exp (^loglog^Qmax) for some 6 > 0. (63) 

Then, 

P Wj{T + s) > exp (log^/^ > 1 - o(l), 

where s := fi'^"^-' (VF/20) • log""*"^ Qmax- Here o(l) is uniform over choice of any r < /i(x). 

Let i* G argmaxj Qiij). For any node j, one can construct a path ji = i* , j2, ■ ■ ■ , jn = j of 
length n by allowing repetition. We recall the definition of Ck and Tk- 

Ci = exp (log log^/"' Qmax ) and Ck = exp ( log^/"^ ) for k>2 



To = and Tk = Y^ g("i)(£i/20) • log"+3 q^^^. 

1=1 

Also define (Bk as 

^fc := {X(r + rfe):T^,,+,(r + rfc)>£fc+i}. 
Now consider the following proposition: 

Proposition 18 For k = 1, . . . ,n — 1 and r < h{x) — Tn-i, 

Pfgfc I > l-o(l). 

In above, o(l) is uniform over choice of t < h(x) — Tn~i- 

Proof. We will prove Proposition [18] by induction. The base case k = 1 follows from 
W,,{t) = Wi*{T) > log Q,.(r) > log(Qi*-r) 

> log(Qmax - /l(x)) > log(Qmax " o(Qmax)) 

> 2 Q™*^^ - log^/^ Qmax) 
= A, 

where inequalities hold for large enough Qmax- It is easy to establish the necessary induction 
step using Proposition [T7] and Ck = exp (loglog"^ Qmax) with 6 = 1/4^^. This completes the proof 
of Proposition [T51 □ 

Therefore, Proposition [18] implies that for r G [0, h(x) — Rc], 

F [Wi{T + Rc) > exp (log log" Q^ax)] = P [Wj{T + Tn-l) > Cn] 

= r[Wj^{T + Tn-l)>Cn] 

= P[£n-l] 
n-1 

= nP[efc I 

k=l 

> (i-o(i)r-^ 

> l-o(l). 

Note that in above, n is a constant and o(l) is with respect to scaling of network state such as 
Qmax- This completes the proof of ([62]) . 
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7.2 Proof of Lemma I 

We first state the following key proposition for the proof of Lemma [T3l 
Proposition 19 // C(x) < W^,^^, then 

F[A^{r)>k\ = exp(--^).0(T), 

for T < /i(x) and k > g^^^'^ (^max) • Recall that Wmax is defined as per (j48p . 

Proof. First note that it is enough to consider the case when k is an integer. Let random time 
To = < Ti < T2 < . . . such that is the mth time when is updated i.e. Bj{Tm — 1) > 2 
and B'j{Tm) = 0. We define 

m := inf |m : Bj{Tm — 1) > g{k — 1) and ?n > l| . 
Now observe that if > k, then 

A]{t) > k > A] 

since A) < g^-^'^{C{x)) < g^'^^ (VTj^ax) < ^- Hence, if A){t) > k,Tff,<T since there should be 
at least two updates before time r which make Aj{-) increase beyond k — 1. Otherwise, Aj{-) 
should keep less than k + 1 under the algorithm until time r. Using this observation, we have 

m](.T)>k] < P[T^<r] 
1=1 

(b) ^ 

< 2l,^P["i('5) = 1 for s = ^ - 2, . . . , / - 1) - 1] 

1=1 



< 



Wmax 



1 

< r • 1 



max 



^ ' I'max ^ 

where (a) is from the union bound; (b) is from m > 1; for (c) one can observe that under the 
algorithm the probability that some node j keeps attempting to transmit consecutively (without 

stopping) for some time interval of length y is at most I 1 — taj — ) • This completes the proof 
of Proposition [191 D 
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Completing proof of Lemma 1131 We derive the following inequalities. 

oo 

E[4(Mx))^] = ^mmx)) = k] . k' 

k=l 

■yAW-l oo 

= nA]{h{^)) = k]-k^+ j2 mihix)) = k].k' 

oo 

< h{x)+ nAi{h{x)) = k] ■ e 

(") ^ 1 

fc=-y/h(x) 

OO ^ 

< h{x) + 0{h{x)). E fc2 = 0{h{x)), 

k=^h{x) 

where for (a) we use the following inequality: 

P[^(^(x)) = k] < • 0(/i(x)), for k > ^/h(x). (64) 

Hence, it is enough to show (I64p to complete the proof of Lemma [131 
Prom Proposition 1191 it suffices to prove that 

, 9{k) \ ^ 1 
exp ^ 



By taking the double-logarithm (i.e. log log) on both sides of the above inequality and using 
g{k) = exp (loglog^ /c), we have the equivalent inequality as 

log log'* k - log Wmax > log 4 + log log k. 

One can check the above inequality holds if log log^ k > 2 log Wmax since Wmax is large enough. 
Equivalently, the desired condition for k is 



k > exp(exp((21ogWI„ax)^/^ 
Pinally, k > \J h(x) satisfies the above condition since 

exp (exp ((2 log Wn,ax)'/^) ) = exp (exp (g ( log logV^ 

< ■ Q log^/^ Qmax) ^ 

where the first inequality is from the definition of Wmax in (|48p and the second inequality holds 
for large enough Qmax- This completes the proof of ()64p . hence the proof of Lemma [T3l 
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7.3 Proof of Lemma 

To begin with, we note that the proof of Lemma [T3] is almost identical to that of Lemma [7] in 
Section [6.2[ Let the random time r* = inf{r : aj(r) = 0} i.e. the first time when j does not 
attempt to transmit, and the event (B denotes r* > /i(x). Hence, if £ happens, B{h{x)) = B+h{x) 
and 

¥[<B] < ¥ [j attempts to transmit consecutively for time r G [0, /i(x) — 1] ] 

^ ' •'max ^ 

where the last inequality because Wj^t) is uniformly bounded above by Wmax- 

On the other hand, if the event (S does not happen, j stops its transmission before time h{x), 
hence -Bj(-) should set to before time h(x). Based on this observation and arguments similar 
to those used for establishing (f65]) . we obtain 

F[Bj{h(x)) = k I G;^] < P[j attempts to transmit consecutively 

for time r G [h{x) — k, h{x) — 1] ] 
/ 1 \ k-l 

< ■ (66) 



Further observe that 
E 



-j^-'HBih{x)))\ = P[e:] •E[5("^)(i?j(/i(x))) I ^]+¥[^'].E[g^-'\B}{h{x))) \ C:^ 

< P[e:] •E[g(-^)(i?i(/i(x))) I +E[5(-^)(i?j(/i(x))) I e]. (67) 

For the first term in (|67p . we consider the following using (j65p . 



¥[<a]-E[g^-'\B}{h{x)))\iB] < [l - -L_Y^'^~\ gi-^) ^b} + h{x)) 

^ ('-Sm) ■»'-"('.(x) + /.(x)) 
S 0(1), 

where (a) follows from Wmax ^ 2 log Qmax 

< /i(x) and < C(x) < Wl,^^ = log^ Q^^^ < h{x); 

one can check (b) for large enough h{x). 

For the second term in (|67p . we consider the following using (j66p . 

Eb(-^)(Bj(/.(x))) I r] < j;5^-^)(fc) • (l - ^) " 

fc=i 

o(,(-)(wLx)) 

< o(5(-^)(41og2Qmax)), (69) 

where (a) is from Proposition [TTJ Combining (j67p , (|68p and (|69p completes the proof of Lemma 
M 
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8 Discussion 



As the main result of this paper, we presented a new medium- access algorithm for an arbitrary 
wireless network where simultaneously transmitting nodes must form an independent set of the 
network graph. The algorithm is optimal in the sense that network Markov chain is positive- 
recurrent as long as the imposed traffic demand can be satisfied by some scheduling algorithm. 
The algorithm is entirely distributed: the only information it utilizes is its own queue-size and 
the history of collision or successful transmission. In a sense, this work settles an important 
question that has been of interest in distributed computation, communication, probability and 
learning. 

The algorithm we presented builds upon the work of [18] where the algorithm required a 
bit of information exchange between neighbors per unit time. Specifically, the key technical 
contribution of our work is to get rid of this requirement by means of designing a novel learning 
mechanism that essentially estimates the rate of a Bernoulli process with time varying rates. 
This learning mechanism could be of much broader interest. 
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A Properties of Markov Chain in Section 14.11 



A.l Proof of Lemma [2] 

Consider the Markov chain P with weights W G M" such that Vl^min ^ 1- Starting from (0, 0), 
from the description of the Markov chain, it follows inductively that Markov chain is always in 
state (<T,a) G so that a < a component- wise, i.e. for any f, <Tj = 1 =^ = 1. Further, 
transition from any such state x = (cr,a) G ^2 to x' = {cr',a') G is possible if and only if 
(a) <T U cr' G and (b) for any i, = 1 a'j =0, V j G From (a) and (b), it 

immediately follows (a') <t U <t' G I{G), and (b') for any i' , a'-,=l =^ aj/ = 0, V j' G M{i'). 
That is, if transition from x to x' is feasible, then transition from x' to x is feasible. As per 
this, it immediately follows that the state (0, 0) is reachable from (and to) all feasible states. 
That is, starting from (0, 0) the Markov chain P is recurrent and let VL' be the recurrent class. 
Further, there is a strictly positive probability of being at state (0, 0). Therefore, Markov chain 
is aperiodic on Vt' . More generally, it can be checked that for any two states x, x' with positive 
Pxx'j it is equal to 

p^. = c(x,x')^ n n (i-i^). (™> 

where c(x,x') = 2"!^^ ''^=^'^ <=^}\. 

Now to establish (jl2p . consider another Markov chain Q on $7' with Qxx' > if and only if 
-fxx' > 0. Specifically, for x = (cr, a), x' = (cr', a') G Vl' with x 7^ x' and Pxx' > 0, 

-3-=^- n n (i-T^)- pi) 

ie<T\<T' iecrncr' 

We choose the other 'self-transitions' in Q so as to make it a valid probability transition matrix. 
This is indeed possible since Qxx' ^ -Pxx' for all x 7^ x' from ()70p . By definition Q is recurrent 
and apreiodic since P is. Therefore, it has unique stationary distribution, say ^. We claim that 
for any x = (cr, a) G ^2', 

iia^a) OC n = ('^ • log ^) = ^('^)- (72) 

This is because, the following detailed-balanced condition is satisfied by Q, ^: for any feasible 
transitions x = (cr, a), x' = (cr', a') G il' with x 7^ x', 

Qxx' ^ nigo-\g-' WT ^ I\iecr'\a- ^ Higo-' ^ ^x^ 

Qx'x ~ n»g^'\^ Wl ~ ~ W.ea W^ " ^x ' 

Given characterization ^ and similarity between Q and P, we shall approximately charac- 
terize TT, the stationary distribution of P, in form of For this, we shall use the following 
proposition. 

Proposition 20 Given a finite state space S, denoted by {1, . . . ,N}, consider two irreducible 
and apriodic Markov chains on S with transition probability matrices A and B. Let Aij > if 
and only if Bij > for all i,j G S. Define 

P(Ai?)= max (^,^). (73) 
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Let 7v^ and be stationary distributions of A and B. Then, 

< max - 

^ TT,- / i V TT; 



R{A,By^ <min (^) < max (^) < R{A, B)^ . (74) 



Subsequently, the relative entropy between tt^ and tt^, denoted by D{7v^\7v^), is bounded above 
as L>(7r^|7r^) < NlogR{A,B). 

Let TV be the stationary distribution of P. For each x G fi', tTx > 0. Therefore, we can write 

TTx oc exp (^F{xfj , (75) 
with F -M' with F((0, 0)) = 0. Now from definition of P and Q, it follows that 

i?(P,Q) < 2". 

Using this, Proposition [20] and the form of $, (cf. ()72p ). we have 

exp (f(x)) = ^ 

/ \ TTx C(0,0) 

= exp I cr • log vr I - — — - 

^ ^ ?x ^(0,0) 

= exp (a ■logW + C/(x)) , (76) 



where |C/(x)| < log R{P,Q) < n4"log2. That is, we conclude that for any x G 0' 

TTx oc exp (^F(x)^ , (77) 

where F{x.) = a ■ log W + f/(x) where |^7(x)| < n4" log 2. This completes the proof of Lemma [2j 

Proof. (Proposition I20p The proof follows by use of characterization of stationary distribu- 
tion by means of Markov chain tree theorem (cf. [3]). Specifically, it characterizes the stationary 
distribution of a finite state, irreducible and aperiodic Markov chain, say A, as follows. Let 
Q = (S,£') be a directed graph with e = (i, j) G C S x S if Aij > 0. Then its stationary 
distribution, tt"^, is characterized as 

TT^^OC ^ n (78) 
T&T{i) {k,l)<^T 

where T{i) is the set of all directed spanning trees of Q rooted at i; by (k, i) G T we mean 
directed edge {k,i) that belongs to T. 

As per hypothesis of Proposition, it follows that the transition graph Q for Markov chains A 
and B are identical. Therefore, from ()78p it follows that for any i G S 

7rf_ ^ T.TeT{i) U{k,e)€T ^ke ^^^^ 
7i"f YlreTii) ll(k,e)€T ^ki 
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By definition of R{A,B), the fact that number of edges in any tree T is N 
follows that for any i G S 



ISI and (Hal) it 



R{A,B) 



< 



TT 



B 



< R{A,B 



(80) 



To establish bound on relative entropy of tv^, tt^ , observe that 



<Y^7TflogRiA,By 

i 

= {Y,^t)N\ogR{AB) 

i 

This completes the proof of Proposition [20l 



N\ogR{A,B) 



(81) 
□ 



> TTmax exp 
1 

> exp ( - nVTmax 



> ^exp 



A. 2 Proof of Lemma [5] 

By definition of TTmin, T^max and from (fT2]l . it follows that 

TTmin > T^max exp ( - JlVFmax " "4" log 2) 

nWr^^^-nA^ log 2) 

n4"log4) = (82) 

where we have used < 4", nlog2 > 1 for n > 2 and C„ = exp(— n4'^ log4). 

Let the time-reversal of P be P* , i.e. P^^, = 7rx'-Pxx'/'''"x for any x, x' G O'. It follows that 
PP* is a reversible Markov chain on 17'. Then PP* has real eigenvalues taking values in [—1, 1]: 
let they be — 1 < Amin < • • • < A2 < Ai < 1. It can be checked that PP* is irreducible and 
aperiodic due to structure of P. Therefore, it follows that Ai = 1, Xpp* = maxjlAmml, A2} < 1 
and PP* has the unique stationary distribution equals to tt, the stationary distribution of 
P, which corresponds to the (normalized) eigenvector with eigenvalue 1. In this setting, the 
following is a well known [161 Corollary 1.14] bound on 'mixing time' of P: starting from any 
initial distribution fj, on fi'. 



1 



TV 



< e, for r > — 

2.77 1 



XpP' 



log' 



1 



(83) 



where the (chi-squared) distance between two distributions on a finite state space (here O') 
is defined as 



1 



2,M 



- Mll2,i 




(84) 



Another distance of interest is total-variation, which is defined as and related to distance as 
follows. 

> III/ — ij-Wtv 



^-1 



2,/i 



(85) 



41 



From (I83|) and ([85]), it follows that 

Wt^P^ -tvWtv <e,for T>- log-^. (86) 

i — App* ^eTTmin 

Thus, to bound the 'mixing time' Tmix of P, we need an upper bound on App*. To that end, 
we shall bound the second largest eigenvalue A2 and the smallest eigenvalue Amin in that order. 
For A2, by Cheeger's inequality [HHI] it is well known that 

$2 

A. < 1--, 

where $ is the conductance of PP*, defined as 

. Q{S,S'^) 
<p = mm — 

Scn':n{S)<^ 7r(S')7r(S'c) ' 

where = Q'\S, it{S) = J2x£S and 



Therefore, 



<!>> mm Q( 3,3") > min TTjPP*)^y 
sen (pp*)^y^o 



> TTmin min (PP*)xy 

MPP*)xy^O 



>7rmin( min -PxO-PyO— ), where = (0, 0) 
V(pp*),y^o ^ vro/ 

> CnW-l, X (2W^ax)-'" exp ( - n4" In 2) 

>ClW-t, (87) 

where (a) is from (j82p . definition of vr from (|12p and definition of Cn = . Now for Xmin, 

we observe that for any x £ Q', 



{pp*U>pi,- 



'Hi 

> (2iy^ax)-'"exp(-n4"ln2) 

Now it can be easily checked that Amin > — 1 + 2 minx(-PP*)xx- From this and ([57j) . it follows 
that 

App* < 1 - lc-4^-6n 

^ 2 max- 

Using (j82p . ()86p and (j88p . it follows that starting from any initial distribution n on Q' , \\nP'^ 
nWxv < £ for 

T > Tmix = ^C-'W'Z. log (^^7^ 
This completes the proof of Lemma [3l 
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B Properties of W{-) 

Here we establish deterministic properties of W{-) under the algorithm that will be useful to 
prove the main theorem in this paper. Specifically we establish that, for any i, Wj(-) changes 
slowly if it becomes large. 

Proposition 21 There exists an absolute constant Wq > so that for any node i and time t, 
if Wi^r) > Wo then 

\W,{r + 1) - W.{t)\ < L,,, (89) 

' g( ^> (exp (log Wi{T))) 

Proof. We shall establish existence of large enough constant Wq under which the claimed result 
holds. To that end, given node i and time r, from definition of weight as per ([T]), 



Wi{T) = exp ( max{/(Qi(T)), / max log ^(^Ut)) 

where we use f{x) = loglogx. Now Qi{-) and ^j(-) for any j G M{i) changed by at most ±1 in 
unit time. That is, they are uniformly 1-Lipschitz. Therefore, 



W,{t + 1) < exp ( max{/(Qi(r) + 1), ^/ .nmx log5(4(r) + l)}j, (90) 



W,{t + 1) > exp max{/(g,(r) - 1), / max logg(Aj(r) - 1)} . (91) 

Using (j90p . we shall establish an upper bound on Wi(r + 1) — VFj(r). To that end, consider 
exp(/((5j(r) + 1)): using Taylor's expansion 

exp(/(Qi(T) + l)) <log(Qi(r) + l) 

1 



< log Qi(r) + 



Qi{T) 



where (a) follows from the fact that logy + 1/y > logx + l/a;ifO < x < y for all y large 
enough; Wq is chosen so that such is true when y > Wq] the fact that logQj(r) < Wj(r) and the 
assumption that Wj(r) > Wo- In a similar manner, using Taylor's expansion and the form of the 
derivative of function exp( y^log g{-)), we have 

1 2 log log 4 (r) 



exp UloggiA]{T) + l)j < exp Ulogg{A]{T)) 



1 + 



4(r) log4(T) 



< W,{t) + W,{t)— (93) 

g(-^)(^exp{log^Wi{T)) 



In above (a) follows because for all large enough y, q{x) < q{y) for < x < y with q{x 
I^T^^; for y large enough q{y) < ^ 



1 2 log log X _ y large enough q{y) < ^; is large enough so that these two inequalities are 
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satisfied; Wi{T) > exp(^/log^* (r)) and VFi(r) > Wo. From §0^, ([92]) and ([MD, it follows that 



W^{T + 1) < Wi{T) + max I exp(-VFi(r)), W^,(t) ^| 



<Wi{T) + Wi{T) 



where the last inequality follows from the fact that for x large enough, exp(— x) < x/g^ (exp(log^ 
This is because g^~^\y) = exp(exp(log^/'^ y)), 

log^''^ exp(log^ x) = o(logx), and hence 

(7^~^^ (exp(log^ x)) = exp(exp(log"'^''^(exp(log^ x)))) = exp(o(a::)). 

We shall assume that Wq is large enough to satisfy this and Wj(r) > Wq- 

This completes the proof of upper bound on VFi(r + 1) as desired by Proposition of [211 In the 
process, we implicitly defined Wq'- it is a constant large enough so that (i) logy+l/y > logx+l/x 
if < X < y for all y > Wq] (ii) q{x) < q{y) <l/yforO<x<y with q{x) = | ^^"^^"^^ for all 

y > Wo'^ and (iii) exp(— x) < x/(7^~^^ (exp(log^ x)) for all x > Wq- In a similar manner (details 
are skipped here), an appropriate lower bound on VFj(r + 1) can be obtained (which will lead to 
additional constraints on Wq)- This completes the proof of Proposition [2TJ □ 

Following is an immediate corollary of Proposition [2TJ 

Corollary 22 There exists a large enough constant wi so that for any node i, if Wi{0) > wi 
then 

mr)-wm\< 



<,(-l)(exp{log=»'j(0)) 

for T < g^~^^ i exp (log^ VFi(O)) j /2Wi(0). Subsequently, for any W2 > wi, 



Wi{T) <W2 + l, if Wi{0) < W2, 
Wi{T) >W2-1, if Wi{0) > W2, 



for 2r < g^ ( exp ( log^ W2) ) /w2. 



(94) 
(95) 



Proof. We need to establish existence of large enough constant wi so that claimed result holds. 
For this, we start by constraining wi > Wq, where Wq is the constant from Proposition 1211 In 
addition, we shall assume that wi is large enough so that for all y > wi, g^~^\exp{\og'^ x))/x < 
g^~^\exp{log^ y)) / y as long as < x < y. Such is a possibility since g^~^\exp{log^ x)) scales 
much faster than x. Further, function (7^"^) (exp(log^ x))/x = exp(o(x)). Therefore, it can be 
shown that for large enough x, 

^ ^ g("^(exp(log^x)) ^ g("^)(exp(log^(x- 1))) 
2x ~ X — 1 
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We shall assume wi is chosen to be such large enough constant. Now applying Proposition 121^ 
starting with Wi(0) > wi, it follows that 

\wM-wm\< — 

9(-l)(exp{ VWj(O) 



As per the above bound and choice of wi, \Wi{l) — Wi{0)\ <^ 1. By repeated application of 
Proposition [21] till the summation of the right hand side of the above bound remains less than 
1, we obtain 

\w.{t)-wm\<E — ^'^'^ 

' ' ^ n(-l) 



exp(log2t^i(s) 



Now for all such s, since — Wi(0)| < 1, Wi{0) > wi and above discussed properties of 

wi, g^~^^ we obtain that for all such r 

mr) - wM\ < r — ^^m^ 

<7(-i)(exp(log2(VF,(0)-l)) 
^ , 21^,(0) 

<7(-^)(exp(log2VF,(0) 

Therefore, it follows that the above holds true for all r such that 

g(-i)(exp(log2VF,(0)) 



T < 



The remaining consequences (j94p and (I95p follow immediately from this. This complete the 
proof of Corollary [22l □ 
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C Proofs of Auxiliary Results 

C.l Proof of Proposition 1111 

Using elementary calculus, it follows that 

lim =0 and lim -^'^'f = 0. 

Hence, there exists a constant Ci > such that for x > Ci, 

c/("^)(x) < exp and x ■ g^-'^\x'^ / i) < g'^~'^\x'^) . (96) 

If A; > it follows that 

g^-^\k) < exp(yk/8^ < exp(p-A:/4) = (exp(p/4))'' < (l+p/2)^ (97) 

where the last inequality holds from exp (x) < 1 + 2x for x £ (0, 1). Using this. 



k=l 



p-V4 oo 

k=l fc=p-2/4+l 

oo 

O {g^"'^ (p- V4) • P~') + E (1 + P/'^)' ■ (1 - P)' 

k=p~^/i+i 



fc=p-2/4+l 

O (p-V4)-p-i)+0(p-i) 



where (a) is from ([97|) and for (b) we use (j96|) under assuming p ^ > Ci. Otherwise, note that 
(b) is trivial since bounded above by constant Ci. This completes the proof of Proposition 

m 



C.2 Proof of Proposition [T5] 

We recall the formula dTOl). 



ry J_. rr 



Pxx' — c(x, X 

where c{x^x') is some constant independent of = [Wj]. Hence, we will consider P^x' as a 
real- valued function in several variables {Wi} i.e. Pxx' = Pxx'{^)- 
Now from the mean value theorem in several variables, 



\pL'-PL'\ = \VPxx'{-)-{W^-W^ 

< ||VP,,.(-)ll2-||W'-^^'||2. 
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Using this and (jTOj) . the desired conclusion fohows since one can easily check that ||VPa;a;/(-)||2 
0(1) since each component of W is always at least 1; \\W^ — W'^\\2 = O (maxj \ Wl — W^\). 



C.3 Proof of Proposition I 

Observe that the definition of distribution v implies that for any x G 0, 

T{x) = logZ + logZ^a;. 

Using this, for any distribution // on we obtain 

X X 

= ^fJ'xilogZ + log ) - ^ l^x log fix 

X X 

= ^f^xlogZ + ^ f^lr, log — 



fJ-x 

log ^ + ^ Atx log — 



< logZ + log( J^/i,^ 



= log 2" 

with equality if and only if fx = v. To complete other claim of proposition, consider x* E 
argmaxT(x). Let /i be the Dirac distribution dx*- Then, for this distribution 

Fip)=Tix*). 

But, F{i') > F{fi). Also, the maximal entropy of any distribution on ft is log Therefore, 

T{x*) < F{u) 

= nT{xu)] + HER{u) 

< E[r(x^)] + log|f]|. (98) 

Re-arrangement of terms in (|98p will imply the second claim of Proposition [161 This completes 
the proof of Proposition [161 
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D Proof of Proposition [T7] 



Recall that the Proposition [T7] assumes that C(x) < VTLx- As per the statement of Proposition 
[T71 we wish to prove that 



Wj{T + s) > exp (log^/^w)] > 1 - o(l), 



for choice of s such that 



^(-1) {W/20) ■ log"+^ Q 



n+3 , 



(a) 



< 5^"^^ (1^/20) -exp (^(n + 3)log^/'^W^) 



(b) 5(^1) (exp(log2iy)) 
- 2W ' 

where (a) is from the condition W > exp (log log^ Qmax) and one can check (b) for large enough 
W (depending on 6). 

Now for the case of j = i, from above and Corollarv 1221 we have 

Wj{T + s) > W-1 > exp (log^/^ w) , with probability 1, 



where the last inequality holds for large enough W. 
Now consider the case j 7^ i. In this case, we have 



Wj{T + s) > exp (^logi/^ 



(a) 
> 



> 



exp (^^log5(^^(r + s))j > exp (log^/'^VF 
g{A{{T + s)) > expflog^/2|^ 



giA{{T + s)) > W/20 



> l-o(l), 

where (a) is from definition of Wj; for (b) we use exp (^log^/2 < W/20 for large enough W: 
(c) is due to the following lemma. This completes the proof of Proposition [T71 

Lemma 23 Consider given i, j G A/'(z), W > and network state X(t) = {Q(t), <T(r), ot(T), A(r), B(t)} 
at time r < h{x). Suppose that Qmax is large enough and 



Then, 



Wi{T) > W > exp (^log log'' Qmax) for some 5 > 0. 



g{Ai{T + s)) > W/20 > l-o(l), 



(99) 



where s = g^-^^ {W/20) ■ log"+^ Q 



.n+3 , 
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Proof. First consider the case when g{Aj{T)) > W/10. 1-Lipschitz property of Al{-) imphes 
that 

g{Ai{T + t')) > W/20, for all r' < g^-^\W/W) - ^("^^ (1^/20). (100) 
On the other hand, we have 



iW/20) V'"' To r V °' 20 J 

> exp(exp^logV4-j.-log-/4_j 

= superpolylog(Qmax)- (101) 



where for (a) we use d{x) — d{x/2) > d'{x/2) ■ x/2 with d{x) = exp ^log^^^ x j and x = VF/10; 
(b) is due to > exp (log log^ Qmax) • Therefore, it follows that 

g{A^^{T + s)) > W/20 

since 

s = ^(-1) {W/20) ■ polylog(Q^ax) « g^^^\W /lO) - g^^^^W /20), (102) 

where the inequality is from (jlOOp . (jlOip . and large enough Qmax- 

Now consider the second case when g{Al{T)) < W/\0. As the first step, we will find some 
absolute upper and lower bounds of Wi{T + r') and g{Al{T + r')) for r' < s. Based on these 
bounds, we will construct a martingale with respect to g{A-'^{-)) to control g{A-'^{T + A)), which 
is indeed similar to the strategy we use for the proof of Lemma [6] in Section 16. 1[ 



First step: Bounds for VFj(r + r'), g{Al{T + r')). From Corollary [22l we observe that for 
t' <s 

Wi{T + T')>W -I, (103) 
since using ()102p it is easy to check that 

I , X 9^'^^ (exp(log2VF)) 
t' < s < g^~^>{W /lO) < ^ ^ for large enough VT. 

For the bound of g{A-^-(T + t')), we obtain that for t' < s 

g{Ai{T + T'))<W/5, (104) 
using 1-Lipschitz property of Aj^-) and 

t' < s < g^-^'{W/10) < g^-^> {W/ 5) - g^-^\W/ 10), 
where (a) is from (jl02p and (b) is due to g^~^\x) > g^~^\x/2) ■ 2 for large enough x. 
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Second step: Martingale construction. This part of the proof is similar to that stated in 
Section 16.11 We consider a modified network Markov chain where all the Markovian rules are 
same as the original chain except for W{t') = W{t' — 1) for r' > r + s i.e. W{-) is fixed after 
time T + s. This modification does not affect the distribution of g{Aj{T + s)); it is merely for 
guaranteeing (jl03p all r > 0. 

Now define random time Tq = t < Ti < T2 < . . . such that Tm is the mth time when A^-) 
is updated since time r i.e. Bj (Tm — 1) > 2 and Bj (Tm) = 0. Further, define for m > 0, 



Y ■- 



exp 



g(~^){W)-Ai{Tm)] if Tm-i<T + s or m = 



[a-Ym-i otherwise 

where a = | + ^ G (0, 1). We shall establish that for all m > 1, 

E [Ym+i I J™] < a • y^, (105) 

where J-m denotes the filtration containing Y/^jT^ for < k < m. Note that (|105p is trivial if 
Tm > T + s by definition of Ym- When Tm. < r + s, we observe that 

E [Ym+l I J'm] 



e^pig^~^\W)-A{{Tm.+i)] \ 



= E 

< \ ■ exp (<7("^)(VF) - A\{Tm) - 1) + ^ • exp {g^-^\W) - A\{Tm) + 1 
= a-exp(<7(-i)(VF)-4(r„ 



= a ■ Ym, 
where for (a) we use 



\A\{Tm^,) = A\(Tm)^\\ = nB\{Tm-\)>9{A\{Tm-x))\ 

(6) 

> nH{Tm - 1) > TV/5] = 1 - nH{Tm - 1) < Wlh\ 

Wjh 

>\-Y,nB\{Tm-\) = k\ 
k=l 
W/5 

> 1 — F[i stops to attempt at time Tm — 1] 

k=l 

(^) 1 3 

> 1 - > > -, 

k=l 



where (b) and (c) are from (|104p and (|103p . respectively. 

From (fT05]l . {Zm := Ym/a'^~^,m> 1} becomes a sub-martingale with respect to J^m- If we 
define a stopping time m* as m* = inf{m : Tm > t + s}, 

(a) (b) / , ^ • \ 

E[Zm*] < E[Zi] = E[yi] < • e = exp {g'^~^\W) - A^t) + l) , 
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where (a) and (b) is from the Doob's optional stopping theorem and 1-Lipschitz property of 
AK'). Using the above inequality and Markov's inequality, we have 



Y . 



a 



m* — l 



Zm* < exp ( (W) - A\ (r) + 1 ) • log Q 



with probability at least 1 



log Q, 



l-o(l). 



(106) 



Finally, it follows that 

A\(t + s) = A{{m* - 1) > A\{m*) - 1 

= 9^~^\W) - (5^-^) W - A\{T^,)) - 1 

> 5^"'^ (VF) - (g^~^^ {W) - A\ (r) + 1 + log log Q„ 



(m* — 1) log — 

a 



log"+2 Q 



log - - log log Qi, 

a 



> 5(-i)(VF/20), 

where (c) is due to the choice of s = g^~^\W/20) ■ log'^"'"'^ Qmax and large enough Qmax! (a) 
and (b) hold with probability 1 — o(l) from ()106p and the Proposition [2^ respectively. This 
completes the proof of Lemma [23l □ 

Proposition 24 



m* - 1 > 



log"+'Q 



1-0(1). 



Proof. We start by defining random variable Ur'- 

{1 if AU-) is updated at time r' „ ; r 

^ , for r' G [r + l,r + s]. 

otherwise 

In other words, Ur' = 1 only if (r' — 1) > 2 and (r') = 0. By definition of 11-^' and m* 



m 



1= 5: Ur 

t'=t+1 



Since WmaxiT') < VVmax = 0(log Qmax) for r' < T + s (cf. (08])), the same arguments in the 
proof of Proposition [10] leads to the following bound for the expectation of m* — 1. 



E[m* - 1] = E 

.r'=r+l 

Now we define random variable Z^' as 

T + S 



+1 



Zr' = E 



E Ur 

.t'=t+1 



(Wn,ax)" 



Ur+1, ■ ■ ■ ,U-r' 



n 



log"+^ Q 
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where r' G [r + 1, r + s + 1]. Hence, it is easy to observe that 

Zt-^i = E[m* — 1] and Z^-^^^i = m* — 1. 

Further, {Zr' : r' € [r + l,r + s + 1]} forms a martingale with bounded increment i.e. 
\Zt-' — Zt-i^i\ < 1. Therefore, the statement of Proposition 1241 follows by applying the Azuma's 
inequality to the martingale {Zr'}. □ 
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